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I  MANAGEMENT  SUMMARY 


MANAGEMENT  SUMMARY 


Many  Information  Systenns  (IS)  managers  may  be  heading  for  a  crisis  in 
relations  with  their  key  users  without  even  being  aware  of  it. 

This  report  shows  that  in  a  number  of  critical  areas  IS  management  and  key 
user  management  have  strikingly  different  views  on  the  same  situation. 

While  on  the  average,  users  are  reasonably  satisfied  with  the  services  they  are 
receiving,  and  their  ratings  are  in  line  with  IS  perceptions  of  user  satisfaction, 
as  shown  in  Exhibit  I- 1,  over  80%  of  IS  respondents  rated  user  satisfaction 
either  higher  or  lower  than  did  user  respondents. 

IS  management's  overrating  of  user  satisfaction  is  an  especially  critical 
issue. 

Low  user  satisfaction  is  illustrated  by  negative  comments  received  from 
virtually  all  key  users  interviewed.  Exhibit  1-2  is  a  representative  sampling  of 
these  comments,  with  the  ratio  of  positive  to  negative  comments  preserved. 

In  many  particular  areas  of  IS  services,  both  IS  management  and  the 
management  of  key  users  agree  that  IS  service  is  satisfactory,  as  shown  in 
Exhibit  1-3. 

There  are  two  critical  areas,  however,  where  there  is  a  gulf  between  IS  and 
key  user  perceptions: 
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EXHIBIT  I-l 


INFORMATION  SYSTEMS  PERCEPTION  VERSUS 
KEY  USER  SATISFACTION 


SATISFACTION 


RATING 


Average  Satisfaction 
Of  Key  Users 


3.  7 


•mv.im 


Companies  Where 
Satisfaction  of  Key 
Users  is  Less 
Than  IS  Perception 


Companies  Where 
Satisfaction  of  Key 
Users  is  Greater 
Than  IS  Perception 


I 


3.  3 


4.1 


2.  4 


3.4 


4.  3 


0 


1 


IS  management  view  of  satisfaction  of  key  users 
Key  user  satisfaction 


RATING:  1  =  LOW,  5  =  HIGH 
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REPRESENTATIVE  USER  COMMENTS 
ON  INFORMATION  SYSTEMS  SERVICE 


POSITIVE 

COMMENTS 

•  "No  problems. " 

•  "New  systems  are  better." 

•  "New  MIS  VP  helping." 

•  "Excellent,  very  responsive." 

NEGATIVE 

COMMENTS 

• 

iviij  musx  realize  trie  uircciiuri 

•      ivi  1  o  seemb  lo  rariuoiii,  nu 

is  for  users  to  manage  their  own 

priorities. " 

data  processing.  MIS  not  pro- 

moting technology  to  do  this." 

•    "EDP  sometimes  insensitive." 

• 

"We  use  90%  of  computer 

•    "EDP  must  show  it  is  a  service." 

resources,  but  are  considered 

low  priority. " 

•    "EDP  spread  too  thin." 

• 

"DP  should  be  more  committed 

•    "MIS  people  vindictive,  arbitrary." 

to  user  needs." 

•    "Data  processing  can't  keep  up 

• 

"User  needs  should  be  prior- 

with our  demands." 

itized  much  better." 

•    "Getting  approvals  from  MIS  a 

• 

"MIS  should  have  a  better 

real  hassle.  " 

understanding  of  corporate 

needs. " 

•    "IS  not  skilled  in  project  manage- 

ment, not  results  oriented." 

• 

"Eliminate  the  traditional  auto- 

cratic EDP  structure  and 

•    "DP  overworked." 

operation. " 

•    "No  EDP-user  communications^ 

• 

"Communication  has  to  improve." 

getting  worse. " 

• 

"MIS  dumps  too  much  on  user." 

•     "Users  feel  patronized." 
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EXHIBIT  1-3 


INFORMATION  SYSTEMS  AND  KEY  USER  MANAGEMENT 
AGREEMENT  ON  PARTICULAR  IS  SERVICES 


SATISFACTION 


RATING 


Documentation 


Training 


System 
Friendliness 


Customer 
Service 


System  Quality 


Program 
Correctness 


Meeting 
Schedules 


3.4 


3.  2 


3.  2 


3.  5 


3.8 


3.  9 


4.0 


3.  8 


3.  8 


3.7 


3.9 


3.  8 


3.  5 


3.  2 


1    1  =  IS  Management  Satisfaction 
HI  =  Key  Users  Management  Satisfaction 

RATING:  1  =  LOW,  5  =  HIGH 


5% 
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Terminal  uptime  (Exhibit  1-4). 

Terminal  response  time  (Exhibit  1-5). 

These  differences  in  perception  can  result  in  IS  allocating  either  too  many  or 
too  few  resources  to  customer  satisfaction. 

The  former  is  wasteful,  the  latter  can  be  fatal. 

Another  area  of  concern  is  the  work  request  process: 

IS  is  quite  pleased  with  current  arrangements  (3.7  on  a  scale  of  5)  while 
key  users  are  considerably  less  pleased  (2.8). 

One  of  the  most  disturbing  findings  is  that  IS  management  feels  that  almost  all 
user-IS  problems  originate  with  the  user  (i.e.,  users'  lack  understanding,  do  not 
get  involved,  or  cannot  perform). 

Users,  on  the  other  hand,  see  these  problems  as  arising  from  bad  IS-user 
communications  or  unsatisfactory  IS  service. 

Obviously,  some  IS  departments  may  have  lost  sight  of  their  position  as 
a  service  organization  and  forgotten  that  the  customer  is  always  right. 

Even  more  important,  almost  half  of  the  key  users  believe  that  a  solution  for 
their  problems  will  be  to  decentralize  data  processing  operations. 

IS  managers  should  seriously  consider  implementing  service  agreements  with 
key  users  (sometimes  called  "user  contracts"). 

Not  many  IS  organizations  have  done  so  yet,  but  early  reports  are  quite 
favorable. 
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EXHIBIT  1-4 


TERMINAL  UPTIME  SATISFACTION: 
DIFFERENCES  BETWEEN  INFORMATION  SYSTEMS  AND  USER  PERCEPTIONS 


SATISFACTION 

RATING 

Companies  where 
satisfaction  of  key 
user  is  less  than 
IS  perception 

Companies  where 
satisfaction  of  key 
user  is  greater 
than  IS  perception 

3.  9 

3.1 

4 

3. 

4.  5 

lilt 

0  12  3  4 


I    I  IS  Management  view  of  satisfaction  of  key  users 
Key  user  satisfaction 


RATING:  1  =  LOW,  5  =  HIGH 
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EXHIBIT  1-5 


TERMINAL  RESPONSE  TIME:  DIFFERENCES 
BETWEEN  INFORMATION  SYSTEMS  AND  USER  PERCEPTIONS 


PERCEPTION 


RATING 


Key  User 
Satisfaction 
Lower  Than 

IS  Rating 


Key  User 
Satisfaction 
Higher  Than 

IS  Rating 


f  MMIMM.MMMrM-  

2.8 


4.4 


Rating  by  IS  Management 

Rating  by  Management  of  Key  Users 


RATING:  1  =  LOW,  5  =  HIGH 
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Service  agreements  will  have  to  be  specifically  tailored  for  the  environ- 
ment and  needs  of  each  organization. 

The  report  recommends  a  specific  strategy  to  follow  in  implementing 
service  agreements. 

•         Also  of  assistance  would  be: 

-      -  Greater  movement  of  personnel  between  IS  and  user  departments,  in 
both  directions. 

Modification  or  elimination  of  present  chargeback  systems. 

A  more  formal  process  for  measuring  user  satisfaction.  (Sample  forms 
are  provided  in  Appendix  D.) 


-8- 

©1982  by  INPUT.  Reproduction  Prohibited. 


INPL 


II  METHODOLOGY 
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METHODOLOGY 


APPROACH 


Without  users  there  would  be  no  IS  department. 

A  few  IS  managers  forget  this  and  many  others  lose  sight  of  it  from 
time  to  time. 

Consequently,  the  level  of  service  which  IS  delivers  is  a  subject  of  constant 
importance.  Recent  developments  have  made  this  even  more  important,  such 
as: 

Technology  changes  (e.g.,  on-line  systems,  distributed  processing, 
personal  computers). 

Increased  dependence  on  computers. 

Financial  pressures. 

It  was  with  these  thoughts  in  mind  that  INPUT  Information  Systems  Program 
(ISP)  clients  selected  the  topic,  "Evaluating  the  EDP  Level  of  Service,"  as  one 
of  those  that  would  receive  an  in-depth  examination  in  INPUT'S  1982  report 
program. 
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•  The  approach  taken  has  been  to  contrast  the  needs  and  expectations  of  IS 
managers  and  users.  Two  principal  sources  of  information  have  been  used  for 
this  study: 

INPUT'S  consulting  work  over  the  last  year  where  corporate  EDP 
productivity  and  effectiveness  issues  have  been  addressed. 

A  special  in-depth  survey  of  IS  management  and  key  user  management. 

•  The  survey  methodology  was  to  first  conduct  an  in-depth  interview  with  key  IS 
executives  concerning  their  views  on  service  performance,  perceived  user 
satisfaction,  and  future  plans. 

•  Based  upon  the  information  obtained  from  the  IS  executive  interview,  a  key 
user  executive  from  the  same  firm  was  then  interviewed.  The  questions  asked 
mirrored  those  asked  of  the  IS  executive. 

•  Copies  of  the  questionnaires  used  are  in  Appendices  A  and  B. 

•  Interviews  were  conducted  by  telephone  in  February  and  March  1982. 

B.       RESPONDENT  PROFILES 

•  The  typical  company  interviewed  had  $1  billion  in  sales  in  1981  and  employed 
between  6,000  and  20,000  employees. 

Companies  interviewed  ranged  in  size  from  $30  million  to  well  over  $20 
billion. 

Appendix  C,  Exhibits  C-l  and  C-2,  provide  a  detailed  breakout  of 
respondent  company  characteristics. 
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Most  of  the  respondent  IS  departments  (84%)  reported  supporting  over  three- 
quarters  of  the  company's  business  operations.  -  , 

Companies  interviewed  covered  a  wide  range  of  commercial  enterprises. 
Appendix  C,  Exhibit  C-3,  has  a  detailed  breakout. 

Similarly,  the  key  user  departments  interviewed  covered  a  wide  range 
of  functional  activities. 

The  operations  area  (i.e.,  line  as  opposed  to  staff  functions)  was 
the  one  most  commonly  cited. 

On  the  average,  the  key  department  consumed  25%-35%  of  IS  department 
machine  resources.  For  70%  of  companies  interviewed,  the  key  department 
consumed  between  20%  and  60%  of  IS  machine  resources.  In  most  of  the 
remainder  the  key  departments  were  even  more  important. 

The  typical  IS  department  interviewed  had  the  CPU  processing  power  equiva- 
lent to  a  3033.  No  company  had  less  processing  power  than  a  433 1 -Group  2 
and  several  respondents  had  very  large  installations. 

The  median  IS  budget  was  $6-10  million,  with  one  quarter  of  the 
disclosed  budgets  being  over  $20  million. 

On  average,  over  half  of  the  hardware  resources  of  sampled  companies  were 
used  by  on-line  systems. 

About  two-thirds  of  the  on-line  resources  were  used  by  user  depart- 
ments (range:  30-90%),  and  the  remainder  by  IS  itself. 

The  sizes  of  the  on-line  networks  ranged  widely,  depending  on  the  size 
of  the  company  and  the  nature  of  its  business. 


-  I  I  - 

)1982  by  INPUT.  Reproduction  Prohibited.  INPUT 


About  half  of  programmer  and  analyst  time  is  being  devoted  to  new  system 
development  or  major  enhancements  for  the  majority  (60%)  of  companies 
interviewed. 


The  percentage  devoted  to  new  development  and  major  enhancements 
ranges  from  a  low  of  30%  in  several  companies  to  highs  of  80%  and 
100%  for  two  respondents. 

The  percentages  are  quite  stable  over  time,  being  seen  in  most  cases  as 
unchanged,  whether  looking  two  years  forward  or  two  years  back. 

These  percentages  are  consistent  with  those  reported  in  the  literature 
and  with  INPUT'S  experience. 

Not  surprisingly,  the  key  systems  tend  to  be  older,  as  shown  in  Exhibit  II- 1. 

If  one  assumes  that  systems  whose  age  is  unknown  are  quite  old,  then 
half  of  the  key  systems  are  more  than  ten  years  old. 

The  key  systems  are  receiving  continual  enhancements,  as  shown  in  Exhibit  11-2. 

There  is  an  element  of  chicken  and  egg  in  this: 

Key  application  areas  are  often  those  operating  in  critical, 
dynamic  environments  and  require  the  underlying  data  processing 
system  to  make  similar  changes. 

However,  there  are  also  systems  that  never  quite  get  off  the 
ground  for  any  number  of  reasons  (user  turnover,  fuzzy  or 
changing  requirements,  technical  problems).  These  "problem 
children"  become  key  systems  due  to  the  amount  of  time  they 
consume. 
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EXHIBIT  ll-l 


AGE  OF  KEY  SYSTEMS 


Percent  of  Systems 


N  =  28 
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EXHIBIT  11-2 


TYPE  OF  ENHANCEMENTS  TO  KEY  SYSTEMS 


ENHANCEMENT 

PERCENT  OF  SYSTEMS 

Ongoing 

71% 

Other 

29 

N  =  24 
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The  ongoing  work  is  by  no  means  trivial,  several  companies  mentioned 
major  changes  being  planned.  Equally  striking  are  the  companies  that 
did  not  report  ongoing  enhancements:  almost  three-quarters  of  them 
reported  major  changes  being  planned  in  the  key  system. 

•  Consequently,  in  spite  of  both  the  age  and  growth  of  the  key  systems,  both 
ongoing  and  major  changes  are  the  rule  and  not  the  exception. 

Although  outside  the  scope  of  this  study,  it  does  seem  apparent  that,  at 
least  in  some  cases,  a  considerable  amount  of  this  modification  would 
have  been  unnecessary  if  the  requirements  and  design  had  been  "right 
the  first  time". 

•  Companies  are  beginning  to  use  specific  techniques  for  software  construction. 
Almost  two-thirds  of  those  interviewed  used  some  type  of  special  technique 
for  managing  the  software  process,  as  shown  in  Exhibit  11-3. 

Specific  proprietary  techniques  include: 

Arthur  Andersen  Method  I. 

Cane  and  Sarson. 

Pride/ASDM. 

Spectrum. 

Warnier-Orr. 

Yourdon. 

These  are  described  in  INPUT'S  December  1980  report,  Managing  The 
System  Development  Process. 
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EXHIBIT  11-3 


SOFTWARE  TECHNIQUES  USED 


Percent  Companies  Using 


NOTE:  Where  a  respondent  used  both  standard  techniques  and  its  own 
techniques,  the  company  was  counted  as  using  a  standard 
technique. 


N  =  27 
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Note,  however,  that  half  the  respondents  are  using  their  "own"  technique.  This 
can  cause  difficulties  on  several  levels: 

It  is  unlikely  that  even  the  largest  infornnation  systems  organization  can 
bring  to  bear  the  quality  and  quantity  of  resources  in  developing  a 
methodology  or  technique  that  a  specialist  organization  can. 

Even  where  an  adequate  tool  can  be  gotten  off  the  ground,  there  is  an 
ongoing  need  for  modification  and  refinement  which  may  receive  a  low 
priority.  In  addition,  this  kind  of  development  is  often  the  "baby"  of  a 
single  key  person;  when  that  person  leaves,  the  effort  collapses. 

At  best,  then,  in-house  techniques  are  uneven  and  sometime  things. 

At  worst,  they  describe  a  hodge-podge  of  activities  that  give  manage- 
ment reason  to  believe  the  situation  is  under  control  when,  in  fact,  it  is 
not. 

Whichever  specific  technique  is  being  used,  IS  management  is,  generally 
speaking,  quite  pleased  with  it  (a  rating  of  4.1  on  a  5-point  scale).  Only  two 
respondents  gave  ratings  of  2  or  below. 
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Ill   SATISFACTION  WITH   INFORMATION  SYSTEMS 

LEVEL  OF  SERVICE 


Ill 


SATISFACTION  WITH  INFORMATION  SYSTEMS  LEVEL  OF  SERVICE 


•  This  chapter  will  discuss  and  analyze  areas  of  both  satisfaction  and  dissatis- 
faction found  in  interviewing  IS  management  and  the  management  of  key  user 
areas. 

A.      AREAS  OF  SATISFACTION 

•  IS  management  is,  in  general,  quite  satisfied  with  the  current  levels  of  service 
being  provided  to  its  users  and  believes  that  service  levels  have  improved 
appreciably  in  the  last  two  years,  as  shown  in  Exhibit  1 1 1- 1. 

IS  management  feels  that  users  (key  users  as  well  as  users  in  general) 
also  believe  that  service  levels  have  improved  and  that  they  are  now 
about  as  satisfied  as  IS  management  is. 

The  key  users  interviewed  show  different  satisfaction  levels  than  those 
believed  to  be  the  case  by  IS  management. 

Satisfaction  levels  are  below  both  those  of  IS  management  and 
those  IS  management  ascribes  to  users. 

Key  user  satisfaction  levels  have  not  changed  much  in  the  past 
two  years. 


-  19- 

©1982  by  INPUT.  Reproduction  Prohibited. 


INPUT 


EXHIBIT 


RATING  OF  GENERAL  INFORMATION  SYSTEMS 
LEVEL  OF  SERVICE 


IS  Management         IS  Management  IS  Management            Attitudes  of 

View                     Rating  of  Rating  of                   Key  Users 

Attitudes  of  Attitudes  of 

General  Users  Key  Users 


I    I  =  Current 

I    I  =  Two  Years  Ago 

RATING:   1  =  LOW,  5  =  HIGH 
N  =  17 
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The  differences  in  satisfaction  between  IS  perceptions  and  user  ratings  do  not 
appear  to  be  based  on  different  views  in  specific  areas. 

Both  IS  management  and  key  users  were  asked  to  rate  the  level  of 
service  in  the  following  areas: 

Documentation.  / 

Training. 

System  friendliness. 

i 

Customer  service. 

System  quality.  i 

Program  correctness. 

Meeting  schedules. 

As  shown  in  Exhibit  1 1 1-2,  the  profiles  were  quite  consistent: 

IS  saw  improvement  from  the  past  to  the  present.  (Interestingly, 
IS  managers  saw  only  half  as  much  improvement  on  the  average 
in  these  specific  areas  as  they  saw  when  asked  the  same  question 
in  general  terms.) 

User  managers  saw  a  similar  improvement,  but  starting  from  a 
lower  base. 

As  shown  in  Exhibit  1 1 1-3,  IS  and  key  users  are  also  reasonably  close  together 
when  rating  key  user  satisfaction  with  user  involvement  in  the  system 
development  process;  i.e.,  the  new  system  development  process  in  general  and 
design  reviews  in  particular. 
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EXHIBIT  III-2 


TWO-YEAR  CHANGES  IN  RATING  OF  LEVELS  OF  SERVICE 
IN  SPECIFIC  AREAS  BY  INFORMATION  SYSTEMS  AND  KEY  USERS 


AREA 


RATING 


Documentation 


Information  Systems 


Key  Users 


Training 
Information  Systems 

Key  Users 


System  Friendliness 
Information  Systems 


Key  Users 


3.  8 


3.  9 


Customer  Service 


Information  Systems 


Key  Users 


U.O 


3.  6 


3.  8 


Information  Systems 
Current 
=  Two  Years  Ago 


Key  Users 
Current 

Two  Years  Ago 


RATING: 
1  =  LOW 
5  =  HIGH 

(Continued) 
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EXHIBIT  III-2  (Cont.) 

TWO-YEAR  CHANGES  IN  RATING  OF  LEVELS  OF  SERVICE 
IN  SPECIFIC  AREAS  BY  INFORMATION  SYSTEMS  AND  KEY  USERS 


AREA 


System  Quality 
Information  Systems 

Key  Users 


Program  Correctness 
information  Systems 

Key  Users 


Meeting  Schedules 
Information  Systems 

Key  Users 


0 


1 


Information  Systems 
I    I  =  Current 

=  Two  Years  Ago 


Key  Users 
=  Current 

=  Two  Years  Ago 


RATING: 
1  =  LOW 
5  =  HIGH 
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EXHIBIT  III-3 


SATISFACTION  WITH  USER  INVOLVEMENT 
IN  THE  SYSTEM  DEVELOPMENT  PROCESS 


SATISFACTION 


RATING 


Information  Systems 
Satisfaction 


User  Satisfaction, 
According  to  IS 


User  Satisfaction, 
According  to  Users 


4.4 


I    I  =  1  980 
=  1984 


N:  IS  =  27,  USER  =  17 


RATING:  1  =  LOW,  5  =  HIGH 
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Users'  satisfaction  two  years  ago  was  somewhat  less  than  IS  believed  to 
be  the  case,  but  views  on  the  current  and  future  situation  are  quite 
close. 

Interestingly,  IS's  own  satisfaction  with  the  development  process  is 
consistantly  less  than  perceived  user  satisfaction. 

j.         This  appears  to  reflect  IS  management's  knowledge  of  the  great 
gap  between  current  practice  and  potential  advances. 

•        Looked  at,  then,  from  an  average  basis,  two  statements  can  be  made: 

The  level  of  user  satisfaction  is  reasonably  high. 

There  is  not  a  great  deal  of  difference  between  the  way  that  users  rate 
their  satisfaction  and  IS  management's  view  of  user  satisfaction. 


B.  DISSATISFACTION 

•  There  are,  however,  a  number  of  areas  that  do  not  conform  to  the  picture 
which,  at  first  glance,  seems  to  show  happy  users.  The  problem  areas 
uncovered  include: 

IS  misperceptions  of  user  satisfaction  levels. 

Specific  areas  of  user  dissatisfaction: 

The  work  request  process. 


Terminal  uptime. 


Terminal  response  time. 
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INFORMATION  SYSTEMS  MISPERCEPTIONS  OF  USER  SATISFACTION 
LEVELS 


The  averaged  reports  of  satisfaction  become  very  different  when  individual 
pairs  of  responses  are  examined.  Exhibit  II 1-4  compares  the  individual  ratings 
of  user  satisfaction. 

Individual  user's  satisfaction  was  both  higher  and  lower  than  IS  manage- 
ment believed  to  be  case. 

However,  the  negative  variation  was  almost  twice  as  high  as  the 
positive  variation. 

Both  types  of  misperception  can  cause  problems. 

Where  users  are  more  satisfied  than  IS  management  believes,  IS  management 
may  be  devoting  resources  to  problems  already  solved.  Several  situations  in 
this  survey  reflected,  for  example,  new  IS  management  correcting  bad 
situations;  they  were  unaware  of  the  magnitude  of  the  progress  that  had  been 
made. 

However,  the  opposite  situation  is  much  more  dangerous:  where  IS  manage- 
ment believes  the  situation  is  well  in  hand  when,  in  fact,  the  key  user  is  less 
happy  than  believed.  Since  these  were  key  users  surveyed,  their  declining 
satisfaction  could  cause  significant  problems  for  IS  management. 

The  specific  areas  discussed  below  are  those  where  both  IS  perceptions  were 
faulty  and  where  key  users  showed  dissatisfaction  (sometimes  dangerous 
dissatisfaction). 
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EXHIBIT  III-4 

GENERAL  LEVEL  OF  KEY  USER  SATISFACTION  (DETAIL) 


KEY  USER 
SATISFACTION, 
AS  RATED  BY 

DIFFERENCE  BETWEEN 
IS  AND  USER 
SATISFACTION 

FIRM 

IS 

KEY  USERS 

USERS 
LOWER 

USERS 
HIGHER 

A 

3.  5 

2.0 

-1.5 

_ 

B 

3.  5 

3.5 

- 

- 

C 

4.5 

2.  5 

-2.0 

- 

D 

U.5 

2.0 

-2.  5 

- 

E 

3.0 

1.0 

-2.0 

- 

F 

4.5 

3.75 

-0.7 

— 

G 

3.0 

2.0 

-1.0 

- 

H 

5.0 

4.0 

-1.0 

- 

1 

4.0 

4.0 

- 

- 

J 

2.5 

4.0 

- 

+  1.5 

K 

4.0 

5.0 

+  1.0 

L 

3.0 

4.0 

+  1.0 

M 

3.0 

3.5 

+0.  5 

N 

3.  5 

2.0 

1 .  5 

0 

4.0 

5.0 

+  1.0 

P 

4.0 

4.0 

Q 

4.0 

4.5 

+0.  5 

Average 

3.  7 

3.  3 

-1.7 

+0.9 

TOTAL 

N/A 

N/A 

-12.2 

+  5.  5 

RATING:  1  =  LOW,  5  =  HIGH 
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DEVELOPMENT  WORK  REQUESTS 


IS  is  fairly  satisfied  with  the  process  used  to  handle  development  work 
requests  and  believes  key  users  to  be  a  Inmost  as  satisfied,  as  shown  in  Exhibit 
1 1 1-5. 

Users  are  sonnewhat  less  satisfied  than  IS  nnanagennent  believes. 

Again,  when  the  averages  are  decomposed  into  individual  paired  responses,  it 
becomes  clear  that  there  are  some  quite  unhappy  users  whose  unhappiness  is 
not  known,  as  shown  in  Exhibit  III-6. 

Twice  as  many  users  were  less  satisfied  than  IS  believed. 

Those  users  that  were  less  satisfied  than  IS  believed  averaged  a 
satisfaction  rating  of  only  1.85  on  a  scale  of  5,  compared  to  IS 
belief  that  they  averaged  3.45. 

Clearly,  IS  has  grown  used  to  a  process  that  produces  multiyear  work  backlogs, 
while  many  users  have  not. 

TERMINAL  UPTIME 

Terminal  uptime  is  quite  important  to  key  users:  the  average  importance  of 
terminal  uptime  is  rated  at  4.9  on  a  scale  of  5  by  the  respondents. 

Both  IS  and  user  management  appear  to  have  similar  perceptions  concerning 
terminal  uptime  and  their  satisfaction  with  it,  as  shown  in  Exhibit  III-7. 

However,  the  averages  conceal  significant  and,  sometimes,  critical  dif- 
ferences. Exhibit  1 1 1-8  expands  the  data  in  Exhibit  1 1 1-7  so  that  detail  for  each 
respondent  is  shown. 
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EXHIBIT  III-5 


DEVELOPMENT  WORK  REQUEST  PROCESS: 
INFORMATION  SYSTEMS  AND  USER  SATISFACTION 


SATISFACTION 

RATING 

IS  Satisfaction  with  Process 

3.7 

Key  User  Satisfaction: 

•  As  Perceived  by  IS 

3.  3 

•  As  Reported  by  Key  Users 

2.8 

RATING:  1  =  LOW,  5  =  HIGH 
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EXHIBIT  111-6 


DEVELOPMENT  WORK  REQUEST 
PROCESS  SATISFACTION  (DETAIL) 


KEY  USER 
SATISFACTION 
ACCORDING  TO 

FIRM 

1    1 IX  J  V 1 

IS 

SAT  ISFACT  ION 

IS 

KEY 
USERS 

A 

4 

4 

3.  5 

B 

2 

2 

C 

3.  5 

3.  5 

D 

3 

3 

E 

3.  5 

3.  5 

F 

4 

4 

G 

5 

4 

3 

H 

4 

3.  5 

5 

1 

4.  5 

3 

4 

1 

ii 

H 

3 

2 

K 

2 

2 

4 

L 

3 

3 

5 

M 

4 

1 

N 

4 

3.  5 

5 

O 

4 

4 

4 

P 

4 

3.  5 

3 

Average 
Satisfaction 

3.  7 

3.  3 

2.8 

RATING:  1  =  LOW,  5  =  HIGH 

NOTE:  TO  PRESERVE  ANONYMITY  THE  ORDER  OF  RESPONSES  IS  NOT  THE 
SAME  IN  THIS  EXHIBIT  AND  IN  EXHIBITS  III-4,  III-8,  and  111-12 

N  =  16 
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EXHIBIT  III-7 


TERMINAL  UPTIME: 
EXPERIENCE  AND  SATISFACTION  (SUMMARY) 


EXPERIENCE 
AND  SATISFACTION 

IS 

MANAGEMENT 

KEY  USER 
MANAGEMENT 

Average  Terminal  Uptime 

Percent,  as  Reported  by: 

94.0% 

92.  3% 

Average  User  Satisfaction, 

as  Rated  by : 

3.  6 

3.7 

RATING:  1  =  LOW,  5  =  HIGH 

N  =  15 
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There  are  differences  in  the  uptime  percentage  reported  in  virtually 
every  casej  six  respondents  believed  uptime  was  better  than  IS  and 
seven  saw  uptime  as  worse. 

There  were  similar,  but  not  as  striking,  differences  in  perceptions  of 
satisfaction. 

What  is  striking,  though,  is  the  differences  in  satisfaction  between  users 
who  were  receiving  higher  response  time  than  IS  believed  (satisfaction 
4.5  on  a  scale  of  5)  and  those  receiving  worse  response  time  (3. 1  rating). 

•         Exhibit  III-9  illustrates  how  tight  this  relationship  is. 

The  two  outlined  boxes  contain  three  times  the  number  of  responses  as 
would  occur  randomly. 

IS  management  will  be  condemned  to  work  at  cross-purposes  with  users 
if  it  has  a  different  perception  of  their  satisfaction  than  is  in  fact  the 
case.  Both  alternatives  are  unsatisfactory: 

Putting  unnecessary,  and  often  very  costly,  resources  into  a 
system  when  the  user  is  satisfied. 

Sitting  by,  while  the  user  festers  and  corporate  objectives  are 
not  met. 

A  large  part  of  this  misunderstanding  is  due  to  linked  misperceptions 
concerning  terminal  uptime.  Several  factors  are  involved. 

Different  perceptions  and  definitions  regarding  uptime  (e.g., 
users  may  only  care  about  and  be  aware  of  uptime  at  critical 
parts  of  their  own  work  cycle). 

Different  measurement  methodologies  (or,  more  usually,  none). 
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EXHIBIT  III-9 

CONTRAST  BETWEEN  INFORMATION  SYSTEMS  AND  USER  PERCEPTIONS 
ON  TERMINAL  UPTIME  AND  USER  SATISFACTION 


c 
o 

•  mm 
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LU 
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LU 


1 

0 

5 

2 

1 

1 

4 

0 

0 

WORSE 


SAME 


BETTER 


User  Rating  of  Terminal  Uptime 
Compared  to  IS  Perception 


RATING:   1  =  LOW,  5  =  HIGH 

Example:    The  "5"  in  the  Higher-Better  cell  means  that 
five  of  the  users  who  gave  a  higher  terminal  uptime 
percent  than  IS  gave,  were  also  more  satisfied  with 
terminal  uptime  than  IS  believed  they  were. 
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Failure  to  document  and  publicize  actual  experiences. 

Publishing  only  favorable  statistics,  while  ignoring  the  unfavor- 
able ones. 

A  logically  difficult  and  unsettling  finding  is  to  compare  the  IS  management 
terminal  uptime  percentage  (94%)  with  that  given  to  a  similar  question 
concerning  the  availability  of  the  "computer  system  generally  during  prime 
shift."  This  percentage  was  95.8%. 

Logically,  these  percentages  are  far  too  close  together,  since  a 
terminal  will  usually  be  down  when  the  computer  system  is  down,  but 
will  also  be  down  on  other  occasions  as  well.  Exhibit  111-10  illustrates 
this  relationship. 

This  issue  of  measuring  terminal  uptime  is  a  difficult  one  (see  INPUT'S 
report.  Performance  Measurement  and  Capacity  Planning,  June  1981). 
However,  the  data  above  show  that  it  must  be  addressed. 

RESPONSE  TIME 

Response  time  issues  were  explored  in  a  similar  fashion  to  terminal  uptime: 

Average  key  system  response  time  as  reported  by  IS  management. 

Average  key  system  response  time  as  reported  by  user  management  of 
the  key  system. 

User  satisfaction  for  the  key  system's  response  time: 

As  reported  by  user  management  of  the  key  system. 
As  reported  by  IS  management. 
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EXHIBIT  111-10 

COMPUTER  SYSTEM  VERSUS  COMPONENT  RELIABILITY 
(HYPOTHETICAL  EXAMPLE) 


L<UiVl  r  \J  IN  C  IN  1 

1  UP-TIME 

HARDWARE 

INDIVIDUAL  TERMINAL 

99% 

MODEMS 

99 

LINES 

99 

MULTIPLEXOR 

99 

CONTROLLER 

99 

DASD 

99 

CHANNEL 

99 

CPU 

99 

TOTAL  HARDWARE 

92% 

SOFTWARE 

DBMS 

98% 

TP  CONTROL  PROGRAM 

99 

OPERATING  SYSTEM 

99 

APPLICATION  SOFTWARE 

99 

TOTAL  SOFTWARE 

95% 

TOTAL  PROCESSING  SYSTEM 

871 

•  FIGURES  FOR  ILLUSTRATION  ONLY. 

SOURCE:  INPUT  REPORT,  "PERFORMANCE  MEASUREMENT  AND  CAPACITY  PLANNING" 
JUNE,  1981 
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Taking  the  unweighted  average  of  responses,  users  saw  response  time  as 
considerably  worse  than  did  IS  management,  as  shown  in  Exhibit  lll-il. 
(Eliminating  one  very  high  user  figure  brings  the  user  average  down  from  6.1 
seconds  to  4.1  seconds;  i.e.,  not  too  dissimilar  to  IS  management's  average  of 
3.6.) 

This  makes  the  similar  satisfaction  averages  understandable. 

However,  when  the  figures  are  disaggregated,  as  shown  in  Exhibit  111-12,  quite 
a  different  picture  emerges: 

User  perception  of  response  times  was  generally  different  from  IS 
management  (column  B),  often  strikingly  so  (e.g.,  response  //8). 

As  with  system  uptime,  where  users  believed  response  time 
higher  than  did  IS  management,  users  had  much  higher  satisfac- 
tion than  where  user  response  time  was  worse  than  IS  believed. 

A  striking  finding  is  that  in  no  case  was  IS  management's  assessment  of 
user  satisfaction  the  same  as  the  user's. 

In  most  cases  user  satisfaction  was  lower  than  IS  management 
believed. 

The  correlation  of  IS  misperception  of  response  time  with  misperception  of 
user  satisfaction,  as  shown  in  Exhibit  111-13,  was  as  strong  as  for  uptime.  That 
is: 

Where  the  user  department's  assessment  of  response  time  was  worse 
than  the  IS  department  believed,  user  satisfaction  was  also  lower  than 
the  IS  department  believed. 
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EXHIBIT  lll-n 


RESPONSE  TIME: 
EXPERIENCE  AND  SATISFACTION  (SUMMARY;) 


EXPERIENCE 
AND  SATISFACTION 

IS 

MANAGEMENT 

KEY  USER 
MANAGEMENT 

Average  Response  Time 

(in  seconds)  as 

3.6 

6.  1 

Reported  by: 

User  Satisfaction 

as  Rated  by: 

3.  9 

3.6 

RATING:  1  =  LOW,  5  =  HIGH 
N  =  14 
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EXHIBIT  111-12 


RESPONSE  TIME  EXPERIENCE  (DETAIL) 
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EXHIBIT  111-13 


CONTRAST  BETWEEN  INFORMATION  SYSTEMS  AND  USER  PERCEPTIONS 
ON  RESPONSE  TIME  AND  USER  SATISFACTION 
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Unfortunately  for  the  IS  departnnent,  even  where  the  user  department 
believed  response  time  to  be  better  than  did  IS  management,  there  was 
a  tendency  to  be  less  satisfied  than  the  IS  department  believed. 

As  most  IS  managers  have  discovered,  there  are  more  ways  to 
lose  than  win  concerning  response  time. 

Several  questions  present  themselves  concerning  the  response  time  issue: 

Are  users  and  IS  management  talking  about  the  same  thing? 

How  good  are  either  side's  measurements? 

Are  the  response  time  numbers  real,  or  are  they  just  "satisfaction" 
expressed  in  a  different  manner? 

To  deal  with  these  questions  in  reverse  order: 

It  is  possible  that  there  could  be  a  steady  bias  concerning  satisfaction 
by  users  and  IS  management. 

Satisfaction  is  to  some  extent  a  relative  measure.  Response 
time  is,  or  should  be,  an  absolute.  No  matter  who  is  measuring 
it,  the  same  results  should  be  obtained. 

Accurate  response  time  measurements  should  result  in  user 
and/or  IS  changing  their  satisfaction  ratings. 

More  disturbing  is  that  users  and  IS  management  are  not  drawing  data  from 
the  same  data  base  (assuming  one  exists). 

Response  time  is  by  its  nature  measurable.  It  is  not  necessary  (or 
always  desirable)  to  measure  each  transaction.  However,  a  standard 
repertory  of  test  transactions  would  provide  the  necessary  data. 
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Little  reliable  data  exist.  "Average"  response  time,  by  its  nature  is  not 
very  meaningful.  A  better  measure  is  95th  percentile  response  time; 
i.e.,  the  minimum  response  time  for  95%  of  transactions  of  a  particular 
type.  This  gets  much  closer  to  describing  peak  loading  situations,  which 
'      is  the  real  concern  to  user  management. 

Unfortunately  only  one  IS  respondent  could  provide  data  for  95th 
percentile  response  time. 

However,  before  IS  management  begins  or  revises  a  measurement  process,  it 
must  first  make  sure  that  it  understands  what  the  users'  needs  are.  Many  users 
have,  consciously  or  unconsciously,  a  hierarchical  or  prioritized  sense  of  what 
response  time  should  be.  Sometimes  it  is  a  fairly  straightforward  type  of 
priority;  e.g.,  two-second  response  time  from  10  a.m.  to  noon  and  2  to  4  p.m. 
and  six  seconds  otherwise. 

Often,  however,  users'  needs  are  more  complex,  comprising  time  of  day 
(or  other  cycle  consideration,  like  time  of  day  within  day  of  week, 
season,  etc.)  and  transaction  type  (or,  perhaps,  type  of  user  or  location). 

it  is  useful  conceptually  to  arrange  these  needs  in  an  array,  as  shown  in 
Exhibit  111-14.  This  cannot  be  done  in  isolation,  but  in  consultation  with 
user  management. 

The  goal  of  this  kind  of  planning  exercise  is  to  deliver  the 
response  time  that  users  need.  Certainly  delivering  less  than 
users  need  should  be  avoided,  since  the  penalties  in  user  satisfac- 
tion can  be  substantial.  (For  further  discussion  refer  to  INPUT'S 
June  1981  report,  Performance  Measurement  and  Capacity 
Planning. 

.  ■  After  such  determinations  have  been  made  with  user  manage- 
ment it  is  important  that  this  information  be  communicated  to 
those  actually  using  the  terminals.  Otherwise  terminal  users  will 
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interpret  a  tailored  menu  of  response  times  as  being  erratic  and 
unpredictable.  Terminal  user  feedback  to  user  management  will 
be  negative  and,  as  too  often  happens,  plans  to  make  things 
better  would  result  in  making  things  worse. 

Consequently,  priorities  should  not  be  made  too  complex  from  the 
standpoint  of  any  individual  terminal  user.  To  do  this  requires 
knowledge  of  how  particular  terminals  are  being  used. 

This  sort  of  information  should  already  be  collected  for  capacity 
planning  purposes. 

.  Priorities  should  be  arranged  so  that  from  a  single  terminal 
user's  perspective  there  are  only  a  limited  number  of  classes  of 
response  time;  response  time  classes  should  be  in  any  event 
readily  explicable. 

Priorities  across  the  whole  computing  network  may,  or  course, 
be  made  quite  complex  as  long  as  the  designers  understand  and 
can  communicate  the  underlying  rationale.  Needless  complexity 
or  that  which  brings  marginal  benefits  should  be  avoided. 
Changes  in  the  external  environment  will  make  complex  priori- 
tization obsolete  much  sooner  than  straightforward,  robust 
priorities. 
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IV       INFORMATION  SYSTEMS  -  USER  RELATIONSHIPS 


•  This  chapter  examines  relationships  between  IS  departments  and  user  depart- 
ments in  several  areas,  including: 

Personnel  movement  between  areas. 

User  data  processing  costs  and  chargeback. 

Outside  services  utilization:  extent  of  use  and  approval  process. 

Current  efforts  to  measure  user  satisfaction  by  IS  departments. 

User  "contracts." 

IS  and  user  views  on  15-user  problems  and  solutions. 
A.      PERSONNEL  MOVEMENT 

•  In  the  course  of  the  in-depth  interviews,  IS  management  and  managers  in  key 
user  areas  described: 

The  existing  level  of  personnel  movement  in  both  directions  between  IS 
and  user  areas. 
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The  desirable  level  of  movement  between  IS  and  user  areas  (in  both 
directions). 

Responses  were  classified  into  three  categories: 

None  (i.e.,  no  movement  of  personnel). 

Some  (including  small  amounts  or  "a  little"  movement). 

Significant  amounts. 

Both  sets  of  managers  are  quite  consistent  in  their  assessment. 

About  half  see  no  current  movement  from  user  to  IS  areas  and  about 
three-quarters  see  no  current  movement  in  the  other  direction,  as 
shown  in  Exhibits  IV- 1  and  IV-2. 

Very  few  see  significant  amounts  of  movement  in  either  direction. 

What  is  most  striking,  and  important,  is  that  over  two  thirds  think  that 
at  least  some  movement  is  desirable. 

Over  one  third  believe  a  significant  amount  of  movement  is 
desirable. 

There  are  two  predominant  reasons  given  in  favor  of  increased  movement: 

Better  career  paths  for  IS  staff  (given  by  IS  management). 

A  better  understanding  of  each  other's  problems  (usually  given  by  user 
management). 
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INFORMATION  SYSTEMS  MANAGEMENT'S  PERSPECTIVE 
OF  INFORMATION  SYSTEMS  -  USER  PERSONNEL  MOVEMENT 
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EXHIBIT  IV-2 


KEY  USER  DEPARTMENT  MANAGERS' 
PERSPECTIVE  OF  INFORMATION  SYSTEMS-USER  PERSONNEL  MOVEMENT 


MOVEMENT 
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PERCENT  OF  RESPONDENTS 


Into  IS 
FROM 
User  Areas* 


From  IS 
\  INTO 
User  Areas  ^ 


None 


Some 


Significant 


None 


Some 


Significant 


47 


25 


42 


24 


33 


71 


I  I 


*  N  =  1 7  for  current 
f  N  =  15  for  desirable 

I    I  Current  Amount 
Desirable  Amount 


0     10      20      30      40      50      60      70       80     90  100% 


-48- 

©1982  by  INPUT.  Reproduction  Prohibited. 


INPUT 

ULOS 


•  This  is  not  an  easy  or  painless  process,  however,  which  in  INPUPs  experience 
is  the  main  reason  why  there  is  not  a  higher  level  of  transfers  in  most 
companies: 

IS  often  welcomes  entry  level  personnel,  but  feels  it  does  not  know  how 
to  train  or  integrate  more  senior  user  staff. 

However,  these  more  experienced  user  employees  would  be  the 
most  useful  in  bringing  substantive  experience  into  IS. 

User  departments  too  often  look  at  IS  staff  for  EDP  liaison  roles,  rather 
than  true  line  roles.  For  those  with  substantive  data  processing  skills 
liaison  roles  are  neither  fish  nor  fowl  and  are  often  unsatisfying.  . 

•  It  was  obvious  in  the  course  of  the  interviews  that  many  managers  on  both 
sides  of  the  fence  had  not  given  serious  thought  to  how  to  make  a  user-IS 
transfer  a  reality  in  their  organization. 

B>       COSTS  AND  CHARGEBACK 

•  A  disturbing  fact  is  the  relatively  small  amount  of  knowledge  that  even  the 
key  user  departments  have  concerning  their  data  processing  costs. 

Less  than  one-quarter  were  able  to  estimate  what  their  department 
spent  on  data  processing  services  from  all  sources. 

The  percentage  of  the  department's  total  budget  which  was 
devoted  to  data  processing  represented  a  very  wide  range:  from 
0.5%  for  a  large  division  to  50%  (for  a  personnel  department's 
human  resource  information  system). 
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Another  quarter  used  the  phrase  "funny  money"  to  describe  the  numbers 
which  purported  to  describe  their  expenditures.  They  obviously  did  not 
take  the  numbers  very  seriously. 

The  remaining  half  of  respondents  had  no  idea  of  their  data  processing 
expenses,  typically  because  they  were  bundled  into  corporate  overhead. 

Thus  three-quarters  of  users  report  that  they  do  not  have  good  numbers  to  use 
to  generate  their  costs  of  data  processing. 

This  is  in  spite  of  the  fact  that  three-quarters  of  respondents  report  a 
chargeback  system  in  place,  as  shown  in  Exhibit  IV-3. 

Note,  however,  the  substantial  "funny  money"  component. 

These  chargeback  systems  have  been  in  use  for  an  average  of  7.5  years 
(with  the  range  from  one  to  15  years). 

Under  one-third  of  respondents  reported  that  "chargeback"  dollars  could 
be  used  for  the  purchase  of  anything  other  than  internal  data  processing 
services;  i.e.,  they  were  administrative,  not  financial  dollars. 

Hence,  for  most  firms  chargeback  dollars  have  become  another  form  of 
overhead. 

Even  this  "overhead  allocation"  mechanism  is  misleading  since 
almost  all  chargeback  systems  are  only  allocating  machine 
resources. 

Charging  for  hardware  usage  occurs  because  of  administrative 
convenience  as  well  as  because  the  chargeback  philosophy  origi- 
nated when  machine  resources  were  scarcer  and  made  up  a 
larger  proportion  of  the  IS  budget. 
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USE  OF  CHARGEBACK  SYSTEMS 


Percent  of  Respondents 
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•  In  spite  of  these  significant  linnitations  on  chargeback  systems,  IS  management 
is  extremely  satisfied  with  the  working  of  whatever  system  is  in  place  (4.6  on 
a  scale  of  5). 

IS  management  believes  that  users  in  general  are  quite  satisfied  (4.2) 
and  that  key  users  are  almost  as  satisfied  (3.8). 

•  However,  the  average  key  user  is  not  nearly  as  satisfied  (3.0)  and  some  are 
extremely  dissatisfied;  e.g.: 

"We  are  charged  for  all  of  EDP's  reruns  and  mistakes."  (The  IS 
department  involved  reports:  "Users  get  what  they  pay  for.") 

"Rates  and  algorithms  haven't  changed  for  ten  years."  (IS 
reports  no  changes  planned.) 

"We  have  no  control  over  quality  of  EDP  personnel."  (The  IS 
respondent  had  stated:  "Users  no  longer  have  a  free  ride.") 

•  About  half  of  the  respondents  reported  changes  are  being  planned  or  studied 
for  their  chargeback  system.  In  most  cases  these  plans  appear  to  be 
nontargetted  or  cosmetic  changes. 

This  is  not  unexpected  given  IS's  high  degree  of  satisfaction  with 
current  arrangements. 

C.       OUTSPE  SERVICES  UTILIZATION 

•  IS  and  key  users  are  in  very  close  agreement  on  the  current  process  used  to 
obtain  outside  services. 
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Two-thirds  report  that  IS  approval  is  needed  before  outside  services  can 
be  obtained,  as  shown  in  Exhibit  IV-4. 

Users  generally  reported  few  problems  with  this  process. 

•  Part  of  the  reason  for  the  high  level  of  agreement  and  relatively  few  problems 
is  that  outside  services  are,  with  few  exceptions,  not  seen  as  very  important  to 
either  side,  as  shown  in  Exhibit  IV-5. 

Both  sides  see  a  slight  decline  in  the  importance  of  outside  timesharing 
(with  IS  believing  it  to  be  somewhat  more  important  than  do  users). 

More  important  is  the  sharp  increase  in  the  role  of  personal  computers 
seen  by  both  groups. 

•  The  way  in  which  key  users  were  selected  might  have  biased  the  responses 
somewhat  away  from  those  who  were  significant  users  of  outside  services: 
they  would  sometimes  not  be  key  users  from  the  IS  viewpoint.  However,  the 
convergence  of  views  shown  here  would  indicate  that  this  is  probably  not  a 
serious  issue. 

D.      MEASURING  USER  SATISFACTION 

•  Very  few  (11%)  of  IS  departments  have  a  formal  program  to  measure  user 
satisfaction;  a  somewhat  larger  proportion  (19%)  is  studying  the  question,  as 
shown  in  Exhibit  IV-6. 

A  majority  of  respondents  (59%)  report  some  type  of  informal  process, 
usually  described  as  consisting  of  committees,  meetings,  or  irregular 
surveys. 
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EXHIBIT  IV-4 


EXTENT  TO  WHICH  INFORMATION  SYSTEMS  APPROVAL 
IS  NEEDED  FOR  OUTSIDE  SERVICES 


IS  APPROVAL  FOR 
USING  OUTSIDE  SERVICES 

BY  PERCENT  OF 

IS 

KEY  USERS 

Required 

67% 

63% 

Not  Required 

18 

25 

Varies 

15 

12 

TOTAL 

100% 

100% 

N:  IS  =  27;  Users  =  17 
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EXHIBIT  IV-5 


IMPORTANCE  OF  ALTERNATIVE  SERVICES 
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ALTERNATIVE  SERVICE 

1  D  D  C  M  T 
L.  U  KK  b  IN  1 

CI  IT  1  IDC 

r  U  1  U  K  t 

PI  1  D  D  C  M  "T 
L<  U  KK  t  IN  1 

CI  IT  1  IDC 

rU 1 UKb 

Outside  Processing 

2.1 

2.1 

1.3 

1.1 

Outside  Timesharing 

2.9 

2.6 

2.4 

2.1 

Turnkey 

1.0 

1.0 

1.1 

1.2 

User  Minicomputers /Mainframes 

1.2 

1.2 

1.3 

1.4 

Consultants/User  Department 
Programmers 

1.4 

1.U 
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EXHIBIT  IV-6 


STATUS  OF  MEASURING  USER 
SATISFACTION  BY  INFORMATION  SYSTEMS 


STATUS 

PERCENT  OF 
IS  RESPONDENTS 

Formal  Process  In  Place 

11% 

Studying  a  Formal  Process 

1  9 

Might  Consider  a  Formal 
Process 

10 

No  Need  for  a  Formal  Process 

60 

TOTAL 

100% 

N  =  27 
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•  There  was  relatively  little  interest  shown  by  respondents  in  doing  more  in  this 
area. 

Ten  percent  of  IS  respondents  reported  that  they  nnight  consider 
increasing  their  efforts. 

However,  the  majority  of  IS  respondents  see  little  need  for  changing  (or 
considering  changing)  to  a  formal  process  of  measuring  user  satisfac- 
tion. 

•  This  is  in  contrast  to  the  key  user  respondents  who  were  much  more  positive 
toward  the  idea,  as  shown  in  Exhibit  IV-7. 

i. 

•  Sample  survey  forms  are  included  in  Appendix  D.  ^ 
E.       INFORMATION  $YSTEM4J$ER  "CONTRACTS" 

•  User  "contracts"  are  a  concept  that  is  beginning  to  be  discussed  and,  as  the 
survey  shows,  beginning  to  be  implemented,  as  shown  in  Exhibit  IV-8. 

A  very  small  proportion  actually  have  them  in  place. 

A  somewhat  larger  percentage  have  some  kind  of  formal  process;  often, 
however,  this  is  just  a  "sign-off"  procedure. 

•  About  one-quarter  of  IS  respondents  were  considering  some  form  of  user 
agreements  or  were  favorably  disposed  to  the  concept. 

Almost  half  of  IS  respondents  saw  no  need  for  this  kind  of  process,  for  a 
number  of  reasons;  e.g.: 

Current  review  process  satisfactory. 
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EXHIBIT  IV-7 


USER  VIEWS  ON  MEASURING 
USER  SATISFACTION 


USER  VIEWS 

PERCENT  OF 
USER  RESPONDENTS 

Definite  Need 

41% 

Might  Be  Useful 

18 

Not  Sure/No  Reply 

29 

Would  Not  Be  Useful 

12 

TOTAL 

100% 
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EXHIBIT  IV-8 


STATUS  OF  INFORMATIONS  SYSTEMS  -  USER  AGREEMENTS 


STATUS 

PERCENT  OF 
IS  RESPONDENTS 

User  Contract  Process  In  Place 

7% 

Partial  Process 

19 

Considering 

19 

Positive  Attitude 

7 

No  Need  Seen  by  IS 

48 

TOTAL 

1  00% 
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Organization  too  small. 


Process  too  complicated. 

Haven't  thought  about  it. 

•        This  is  one  area,  though,  where  IS  is  definitely  ahead  of  users: 

Organizations  that  were  implementing  a  user  contract  system  had  done 
so  very  recently,  so  recently  that  user  respondents  were  unaware  of  the 
implementation! 

■"  .  However,  the  IS  managers  involved  felt  very  positive  about  the 
process  so  far  ("Works  wonders,"  "Very  satisfied,"  "Everybody 
wants  one"). 

Most  users  Interviewed  had  not  heard  of  the  concept.  No  negative  ideas 
were  expressed  by  users  toward  the  concept,  however. 


F.       INFORMATION  SYSTEMS-USER  PROBLEMS  AND  SOLUTIONS 


•  IS  and  key  users  were  asked  several  open-ended  questions  to  obtain  their 
opinions  on  what  the  major  IS-user  problems  are;  they  were  also  asked  to 
suggest  ways  to  resolve  these  problems. 

•  IS  management  was  remarkably  consistent  in  seeing  IS-user  problems  as 
somehow  arising  from  defects  in  the  users  themselves,  as  shown  in  Exhibit  IV-9. 

Four  out  of  five  IS  managers  believed  that  users  did  not  understand, 
were  not  Involved,  or  could  not  perform. 
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EXHIBIT  IV-9 


INFORMATION  SYSTEMS  VIEWS  ON 
INFORMATION  SYSTEMS  -  USER  PROBLEMS  AND  SOLUTIONS 


PROBLEM 

PERCENT  OF 
RESPONDENTS 

SOLUTION 

PERCENT  OF 
RESPONDENTS 

User  Understanding 
-General  33% 
-Specific  11 

Increased  Involvement/ 
Education 

30% 

User  Involvement 

22 

Corporate 
Communications 

11 

User  Performance 
(priorities,  cost 
justification,  etc.) 

15 

Chargeback  System 

7 

None  Stated 

8 

None  Offered 

63 

TOTAL 

100% 

TOTAL 

100% 

N  =  27 
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IS  management  had  relatively  few  solutions  to  offer,  but  these  were 
mainly  concerned  with  improving  the  users  (i.e.,  education  in  the 
problems  and  issues  of  data  processing). 

Users,  on  the  other  hand,  divided  into  two  groups,  as  shown  in  Exhibit  IV- 10: 

Those  who  saw  that  IS  and  themselves  just  were  not  communicating. 

_         Those  who  saw  themselves  getting  unsatisfactory  service  from  IS. 

Note:     for  representative  comments,  see  Exhibit  1-2,  in  the 
-  management  summary. 

Users  saw  solutions  relating  to  their  perceived  problems: 

Better  communications. 

Decentralized  data  processing  operations.  (Note  that  these  respondents 
were  not  just  asking  for  improved  existing  operations;  for  these 
respondents,  there  is  a  strong  note  of  having  lost  hope  in  having  their 
problems  resolved  by  "business  as  usual.") 

This  gulf  between  IS  and  users  is  a  serious  one.  The  remainder  of  this  study 
will  address  the  issue  of  how  IS  management  can  rectify  matters. 
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EXHIBIT  IV-10 


KEY  USER  VIEWS  ON 
INFORMATION  SYSTEMS-USER  PROBLEMS  AND  SOLUTIONS 


PROBLEM 

PERCENT  OF 
RESPONDENTS 

SOLUTION 

PERCENT  OF 
RESPONDENTS 

iS-User  Communications 

47% 

Better  Communications 

35% 

Unsatisfactory  Service 

41 

Decentralized  Operations 

-  General  30% 

-  Personal 
Computers  17 

47 

None  Stated 

12 

None  Offered 

18 

TOTAL 

100% 

TOTAL 

100% 

N  =  17 


-63- 

©1982  by  INPUT.  Reproduction  Prohibited. 


INPUT 

ULOS 


-64- 


V  CONCLUSIONS  AND  RECOMMENDATIONS 


CONCLUSIONS  AND  RECOMMENDATIONS 


CONCLUSIONS 


Users  are  far  less  content  and  far  more  willing  to  consider  alternatives  to 
existing  data  processing  arrangements  than  IS  believes. 

While  in  some  respects  users  are  satisfied  with  the  level  of  service  they  are 
receiving,  in  certain  key  areas  they  are  not;  i.e.,  in: 

Terminal  downtime. 

Terminal  response  time. 

Program  development  requests. 

The  areas  with  the  greatest  problems  are  exactly  those  areas  where  users  have 
the  most  contact  with  IS.  Thus,  there  is  no  room  for  the  self-satisfaction  that 
some  IS  management  exhibits. 

Chargeback  systems  are  not  the  panacea  they  were  once  believed  to  be: 
Most  users  cannot  freely  choose  alternatives. 
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In  many  cases  chargeback  systems  are  accounting  artifacts  and,  equally 
important,  are  so  perceived  to  be. 

By  their  nature,  chargeback  systems  measure  and  "charge"  for 
resources  used,  rather  than  value  received. 

Consequently,  chargeback  systems  are  not  self -regulating  mechanisms  which 
will  "give  users  what  they  pay  for." 

A  better  approach  is  that  used  by  some  IS  departments,  which  is  to  obtain  user 
feedback: 

Formally,  through  surveys. 

Informally,  by  contacts  and  meetings. 

Both  approaches  have  drawbacks. 

Surveys  are  often  episodic  and  usually  do  not  involve  user  management 
as  a  partner. 

Informal  contacts  are  very  subjective  and  may  avoid  key  issues. 

Most  importantly,  neither  approach  provides  a  reference  point  to 
measure  progress  against. 

Often,  of  course,  a  user  has  never  analyzed  exactly  what  the 
department's  particular  needs  are.  The  temptation  is  great, 
then,  for  IS  to  guess  what  the  user's  target  is.  Not  surprisingly 
such  guesses  are  usually  wrong. 
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B.       GENERAL  RECOMMENDATIONS 


•  Chargeback  systems  should  be  viewed  as,  at  best,  a  cost  accounting  tool. 
Periodically,  such  data  should  be  used  as  an  input  to  determine  the  cost  of  a 
process  or  product. 

•  If  chargeback  systems  are  to  carry  out  this  limited  function,  the  costing 
mechanisms  must  be  greatly  expanded: 

All  costs,  not  just  hardware  costs,  should  be  tracked. 

Software  maintenance  costs,  especially,  should  be  tracked  by  project 
and  user.  This  is  rarely  done  now. 

Costs  should  be  based  on  resources  denied,  not  resources  consumed. 

•  IS  management  should  strongly  distrust  what  it  believes  user  satisfaction  to  be 
in  the  absence  of  objective,  rigorously  obtained  information. 

•  Surveys  and  other  attempts  by  IS  to  gain  such  knowledge  can  lead  to 
frustration  by  both  IS  and  users: 

For  IS,  because  user  "satisfaction"  can  prove  to  be  elusive  and  mer- 
curial, seemingly  reflecting  last  week's  triumph  or  crisis. 

Users,  on  the  other  hand,  can  view  such  well  meaning  surveys  as  goads 
which  remind  them  of  the  poor  service  they  believe  they  are  receiving. 

•  Similar  efforts  to  improve  communication  will  often  only  make  it  easier  for 
each  side  to  abuse  the  other. 

•  IS  should  resolve  these,  and  a  number  of  related  problems,  by  recognizing  that 
user  expectations  and  IS  performance  should  be  linked.    Too  often  the  two 
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exist,  at  best,  in  separate  vacuums  and,  all  too  often,  are  not  consciously 
examined. 

•  After  examining  current  practice  and  problems,  INPUT  believes  that  an 
initiative  that  would  usefully  serve  many  organizations  is  the  "user  contract" 
or  as  INPUT  prefers  to  describe  it,  the  user  "service  agreement." 

•  User  agreements,  if  executed  successfully,  establish  common  standards  against 
which  performance  can  then  be  monitored. 

This  sets  up  the  dialogue  which  users  desire. 

IS  gets  its  chance  to  "educate"  users. 

Expectations  can  often  be  influenced  and  managed  by  IS. 

A  joint  measuring  process  is  established. 

Regular  reports  and  meetings  provide  early  warning  of  changing  needs 
and  objectives. 

•  The  next  section  is  devoted  to  exploring  the  opportunities  inherent  in  service 
agreements. 

C      USER  SERVICE  AGREEMENTS 

•  There  are  four  aspects  to  implementing  user  service  agreements  ("user 
contracts")  in  an  organization: 

Deciding  whether  service  agreements  are,  in  fact,  suitable  for  a 
particular  organization. 
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Defining  the  agreements'  contents. 

Establishing  the  initial  service  agreements. 

Ensuring  that  IS  has  the  resources  to  support  the  process. 

Each  of  these  points  will  be  discussed  in  this  section. 

DECIDING  WHERE  SERVICE  AGREEMENTS  ARE  ADVISABLE 

Some  organizations  are  bad  bets  for  introducing  service  agreements. 
Examples  of  this  type  of  environment  include: 

Basic  corporate  policies  are  in  a  constant  state  of  flux. 

If  the  corporation  cannot  answer  the  question,  "Who  am  I?",  then 
particular  user  areas  will  have  similar  problems  in  consistently 
enunciating  their  needs. 

IS  itself  will  also  be  buffetted  in  this  sort  of  environment  and 
would  find  it  difficult  to  live  up  to  its  commitments. 

User  management  turnover  is  high  and/or  user  management  is  not 
strong  or  very  political. 

Agreements  cannot  be  built  on  sand. 

The  underlying  business  is  subject  to  marked  fluctuations  that  cannot  be 
predicted  or  influenced  by  the  organization. 

Reaction  and  catchup  ball  are  the  watchwords.  Agreements 
would  not  make  much  sense. 
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This  is  not  to  say  that  an  organization  must  be  rigid  and  unchanging  to  benefit 
from  service  agreements. 

The  best  environment  is  one  that  is  growing  and  dynamic. 

In  addition,  the  corporate  environment  should  be  receptive  to  planning  in 
general. 

The  environment  would  be  very  favorable  if  past  planning  efforts  have 
resulted  in  bottom-line  benefits  that  are  widely  perceived  and 
accepted. 

FACTORS  DEPENDENT  ON  INFORMATION  SYSTEMS 

Some  of  the  critical  factors  for  making  service  agreements  work  are  depen- 
dent on  the  nature  and  capabilities  within  IS  itself.  It  makes  no  sense  to  enter 
into  agreements  which  IS  stands  little  chance  of  fulfilling. 

For  example,  some  "central"  or  "corporate"  IS  departments  are  staff,  planning, 
or  coordinative  groups.  They  often  do  not  have  either  the  detailed  operational 
knowledge  or  means  of  allocating  specific  operational  resources. 

This  kind  of  group  will  find  it  difficult  to  avoid  over-  or  under- 
committing  resources.  It  will  also  find  it  difficult  to  make  resource 
reallocations. 

The  capabilities  of  IS  staff  and  computer  resources  must  be  candidly  assessed. 

Some  IS  organizations  may  be  in  a  rebuilding  phase.  Taking  on 
voluntary  commitments  that  would  result  in  more  failure  is  not  the  way 
to  nurse  the  patient  back  to  health. 


-70- 

©1982  by  INPUT.  Reproduction  Prohibited. 


INPl 


On  the  other  hand,  where  IS  management  is  new  and  has  succeeded  in 
making  internal  improvements,  service  agreements  are  a  way  of  making 
the  new  state  of  affairs  clear  to  the  outside  world. 

Finally,  there  are  some  specific  capabilities  that  will  be  required  to  make  the 
process  work: 

Project  and  time  estimating  skills  will  be  important  for  IS  to  carry  out 
its  contractual  responsibilities. 

An  adequate  system  should  be  in  place. 

Linked  with  estimating  skills  is  the  requirement  to  deal  with  per- 
formance measurement  questions.  As  noted  earlier,  terminal  uptime 
and  response  time  are  two  of  the  most  critical  issues  involved. 

In  many  organizations,  these  measurements  are  performed 
incompletely  or  not  at  all. 

It  may  be  necessary  to  rely  on  user  personnel  for  some  measure- 
ments in  these  areas,  in  order  to  build  trust  and  communication. 

Exhibit  V-l  summarizes  the  effects  of  the  organizational  and  IS  factors  that 
affect  the  advisability  of  introducing  service  agreements. 

USER-RELATED  FACTORS 

Service  contracts  will  not  serve  a  useful  purpose  if  relations  with  a  particular 
user  are  getting  worse  or  are  already  bad. 
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EXHIBIT  V-1 


ORGANIZATIONAL  AND  INFORMATION  SYSTEMS  FACTORS  AFFECTING 
ADVISABILITY  OF  INTRODUCING  SERVICE  AGREEMENTS 


FACTOR 

POSITIVE 
ELEMENT 

NEUTRAL 
ELEMENT 

NEGATIVE 
ELEMENT 

Corporate  Organization 

Dynamic 

Rigid 

Unstable 

Planning  Receptivity 

High 

Low 

Type  of  IS  Organization 

Centralized 

Decentralized 

Central 
Coordination 

IS  Personnel  Capabilities 

High 

Low 

IS  Top  Management  Time  in  Office 

Short 

Medium,  Long 

Hardware  Capacity 

Adequate 

Inadequate 

Telecommunications  Reliability 

High 

Low 

IS  Time  Estimating  Track  Record 

Reliable 

Unreliable 

IS  Performance  Measurement 
Capabilities 

High 

Low 
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There  is  a  minimum  level  of  trust  and  confidence  required  to  make  such 
agreements  work.  If  this  is  lacking,  trying  to  introduce  a  new, 
unfamiliar  issue  will,  at  the  least,  further  complicate  matters  and,  at 
worst,  make  the  user  think  IS  is  trying  to  divert  attention  from  existing, 
well-known  problems. 

However,  where  relations  are  improving,  even  if  still  not  very  good,  a 
service  agreement  can  be  further  evidence  of  IS's  commitment  to 
improvement. 

There  can  sometimes  be  a  thin  line  between  these  two  situations  and  it 
can  be  useful  to  initially  introduce  an  intermediary  (either  from  inside 
or  outside  the  firm)  to  assess  the  situation  objectively. 

•         It  is  important  that  key  users  be  distinguished  from  the  average  and  limited 
users. 

It  is  the  key  users  who  usually  determine  whether  data  processing  will 
be  successful  in  an  organization  and  how  much  the  use  of  the  computer 
will  contribute  to  the  firm. 

Key  users  are  identified  by  a  combination  of: 

The  importance  of  the  user  department  to  the  organization. 

The  proportion  of  IS  resources  (human  and  hardware)  which  the 
user  department  consumes. 

Exhibit  V-2  illustrates  graphically  the  relationships  between  the  three 
types  of  users. 

Generally,  where  a  department  consumes  significant  systems 
resources  and/or  the  department  is  important  to  the  company,  the 
department  will  be  a  key  user. 
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EXHIBIT  V-2 


DISTINGUISHING  KEY  USERS  FROM  OTHER  USERS 


LOW  HIGH 

Information  Systems  Resources 
Consumed  by  User  Department 
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Note,  however,  that  where  there  is  a  pairing  of  extremes  (high 
consumption  -  low  departmental  importance,  or  vice-versa),  this 
is  probably  not  a  key  user. 

Some  IS  organizations  may  find  that  they  in  fact  have  few  key 
users:  their  biggest  customers  are  not  very  important  in  the 
totality  of  the  organization.  Missionary  work,  which  may  or  may 
not  involve  service  agreements,  would  certainly  be  indicated. 

•  Knowledgeable,  sophisticated  users  are  certainly  good  candidates  for  user 
contracts. 

Many  of  the  definitional  and  expectation  problems  caused  by  more 
naive  users  will  be  absent. 

This  kind  of  user  is  also  often  more  involved  with  the  IS  department  and 
data  processing  operations. 

Of  course,  they  cannot  be  ignored  so  easily  when  things  go  wrong. 

•  Where  users  are  increasing  their  use  of  services  not  supplied  by  IS  this  can  be  a 
sign  that  a  service  agreement  is  needed  so  that  they  receive  the  services  they 
need. 

Sometimes  users  go  outside  for  services  that  IS  cannot  supply  at  all 
(e.g.,  certain  external  information  data  bases)  or  cannot  do  feasibly 
(e.g.,  a  personal  computer  on  every  analyst's  desk). 

However,  sometimes  users  look  elsewhere  because  they  believe,  rightly 
or  wrongly,  that  IS  cannot  meet  their  needs. 

Drawing  up  an  agreement  with  the  user  will,  at  the  least, 
identify  whether  this  is  so. 
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Finally,  current  user  satisfaction  as  perceived  by  IS  should  not  be  a  factor  in 
deciding  whether  to  establish  service  agreements.  As  shown  earlier,  IS  is  nnore 
likely  to  be  wrong  than  right  in  this  assessment. 

Exhibit  V-3  summarizes  the  effects  of  these  user-related  factors  that  would 
affect  the  advisability  of  introducing  service  agreements. 

SERVICE  AGREEMENT  CONTENTS 

Service  agreements  should  focus  on  essentials.  The  following  is  not  required: 
Complexity. 
Undue  formality. 
Legalisms. 
Rigidity. 

Service  agreements  could  become  extremely  large,  if  every  element  of  IS-user 
relationships  were  defined.  In  many  cases  this  is  not  necessary.  Where  routine 
operations  (e.g.,  batch  financial  systems)  have  been  operating  smoothly  for 
many  years,  the  adage,  "if  it  ain't  broke,  don't  fix  it,"  should  be  followed. 

The  areas  that  should  be  focussed  on  are  those  areas  that  are  new,  critical 
and/or  have  a  significant  problem  potential.  INPUT'S  research  indicates  that 
for  many  IS  departments  the  critical  areas  will  be: 

Terminal  downtime. 

Response  time. 

Program  development  requests. 
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EXHIBIT  V-3 


USER-RELATED  FACTORS  AFFECTING 
ADVISABILITY  OF  INTRODUCING  SERVICE  AGREEMENTS 


FACTOR 
(Separate  for  Each  User) 

POSITIVE 
ELEMENT 

NEUTRAL 
ELEMENT 

NEGATIVE 
ELEMENT 

Relations  with  User 

Moderate; 
Bad,  But 
Improving 

Excellent 

Moderate  & 
Worsening; 
Bad 

Amount  of  Outside  Services  Used 

Increasing 

Stable; 
Decreasinq 

User  Sophistication 

High 

Low ; 
Meidium 

Importance  of  EDP  to  User 

High 

Low 

Proportion  of  IS  Resources  Used 

High 

Low; 
Medium 

Amount  of  User  Involvement 

High 

Low ; 
Medium 

User  Satisfaction  as  Perceived  by  IS 

Low, 
Medium, 
High 
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The  exact  issues  to  be  dealt  with  will  naturally  vary  from  user  to  user, 
depending  on  their  needs  and  the  status  of  the  application.  However,  there 
will  be  certain  points  that  should  at  least  be  considered  in  every  case. 

Exhibits  V-4,  V-5,  V-6,  are  checklists  of  items  that  should  be  considered 
for  inclusion  in  every  agreement. 

User  agreements  should  be  viewed  as  flexible,  open-ended  documents  with 
provision  for  modification  by  either  side,  based  on  changing  conditions. 

Again,  a  legalistic  or  buyer-seller  state  of  mind  should  be  avoided.  It  is 
important  that  both  sides  be  able  to  give  as  much  warning  as  possible  to 
the  other  when  adverse  conditions  threaten.    Otherwise,  the  tendency 
!      will  be  to  hope  for  the  best  and  postpone  bad  news  as  long  as  possible. 

This  raises  a  related  question  of  whether  chargeback  rates  should  be  included 
in  the  agreement. 

One  of  the  main  advantages  to  using  the  term  "service  agreement"  is  avoiding 
the  word  "contracts"  which  implies  a  buyer-seller  relationship. 

Most  IS  operations  are  not  set  up  in  such  a  way  that  real  money  (or  even 
reliable  pseudo-money)  changes  hands. 

As  indicated  earlier,  most  chargeback  systems  are,  at  best,  accounting 
conventions. 

Unless  the  chargeback  system  is  thought  through  as  least  as  well 
as  the  user  agreement,  it  would  be  a  mistake  to  associate  the 
I  two  in  one  package. 

If  charges  are  not  included  in  the  agreement,  then  what  does  IS  get  out  of  the 
agreement?  Are  promises  being  made,  with  nothing  received  in  return? 
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SERVICE  AGREEMENT  CONTENT: 
TERMINAL  DOWNTIME  CHECKLIST 


Definition     i          hv  tvnp  nf  malfiinrtinn 

—     Local  hardware 

—     C*ommiinip^tion  lin*^ 

-    Central  hardware 

System  software 

-    Applications  software 

• 

Uptime  requirements  by 

—    Time  oeriod  e.a. 

•  Time  of  day 

•  Day  of  week 

•  Season  etc. 

—     Location  faeoaraohv  oraanizationl 

-    Equipment  type 

V 

M  a  f 'f  1 1  nr'tin  n  Kf^nsiJ" 
ivi CI  1 1  u  1 1^ Li<^  1 1    I  c^aii 

-     R esDonsi bi  1  itv     bv  tvoe  of  malfunction 

-  Process 

—     Fnuinmpnt  hackuD  and  deootinn 

• 

Measurement  methodology,  e.g.. 

-    User  problem  logs 

System  monitors 

• 

Measurement  responsibilities 

-  Reporting 

-  Meetings 
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SERVICE  AGREEMENT  CONTENT: 
RESPONSE  TIME  CHECKLIST 


•  Definition,  i.e., 

-  Minimum 

-  Maximum 

-  Average  (mean) 

-  Average  (median) 

-  Percentile  (e.g.  95th) 

•  Requirements  by 

-  Time  of  day 

-  Time  period  (day  of  week,  month, 

season,  etc.) 

-  Location  (geography,  organization) 

-  Type  of  transaction 

•  Priorities  by  type  of  requirement 

-  E.g.,  where  contention/degradation  occur 

•  Measurement  methodology,  e.g., 

-  System  monitors 

-  On-site  stopwatch  sampling 

•  Measurement  responsibilities 

Reporting 
Meetings 
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SERVICE  AGREEMENT  CONTENT: 
PROGRAM  DEVELOPMENT  REQUEST  CHECKLIST 


• 

Definition 

Levels  of  requests 

• 

Process 
—  Forms 

Information 

Routing  to  be  followed 

User  resDonsibi litv 

-  Screening 

-  Prioritizing 

-  Batching  of  requests 

-  Benefit  assessment 
Denein  iiicabur ciiiciii 

• 

1 

Cost/benefit  process 

-  When  to  be  applied 

-  Payback  criteria 

• 

Feasibility  study  role 
Content 

-  Role  of  rapid  prototyping 

-  Turnaround 

• 

Maintenance  "release"  cycle 

-  Time  period 

-  Exception  process 
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IS  management  should  squarely  face  the  fact  that  they  are  an  internal 
service  organization.  All  benefits  are  supposed  to  flow  in  one  direction, 
hence:  service  agreement. 

What  IS  management  receives  is: 

A  baseline  on  which  to  be  judged. 

A  measurement  process  agreed  to  by  key  users. 

A  standardized  process  for  being  informed  of  changes  in  user  activities. 
DEVELOPING  USER  SERVICE  AGREEMENTS 

In  most  organizations  there  will  be  little  initial  pressure  from  users  to 
establish  data  processing  service  agreements. 

As  indicated  in  the  study  findings,  there  is  as  yet  little  user  awareness 
of  the  concept. 

Consequently,  IS  can  introduce  user  service  agreements  in  a  cautious,  orderly 
process. 

Exhibit  V-7  shows  the  typical  steps  that  should  be  followed  in  develop- 
ing user  service  agreements. 

Most  of  the  steps  have  been  discussed  in  previous  sections. 

Make  sure  that  top  management  and  user  managers  understand  the 
concept.  It  is  important  not  to  use  the  word  "contract." 

Be  prepared  to  be  flexible  and,  within  reason,  to  modify  the  form  of 
agreements  to  meet  user  demands. 
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EXHIBIT  V-7 

STEPS  IN  DEVELOPING  USER  SERVICE  AGREEMENTS 


1 .  Decide  whether  service  agreements  are  advisable  in  the 
organization's  environment. 

2.  Develop  and  tailor  a  concept  for  the  specific  organization. 

3.  Classify  users  into  key,  average,  and  limited  use  categories. 

4.  Discuss  concept  with  top  management,  key  users, 
and  selected  average  users. 

5.  Develop  draft  agreement  guidelines. 

6.  Negotiate  a  pilot  agreement  with  one  or  more  average 
users. 

7.  Revise  guidelines  after  pilot  is  in  place. 

8.  Circulate  revised  guidelines  to  key  users;  modify  as 
required. 

9.  Negotiate  agreements  with  key  users. 

10.  Negotiate  agreements  with  remaining  average  users 
after  arrangements  with  key  users  are  working 
satisfactorily. 

11.  Negotiate  agreements  with  limited  users  after  all  other 
agreements  are  in  place. 
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Do  not  promise  more  than  can  be  delivered. 
INFORMATION  SYSTEMS  RESOURCES  NEEDED 

Assuming  that  IS  has  made  the  correct  choice  in  deciding  to  implement  service 
agreements,  then  the  basic  resources  that  IS  needs  to  support  the  process  will 
be  in  place. 

However,  there  will  be  some  specialized  skills  and  tools  that  will  be  of 
considerable  assistance  in  making  service  agreements  work.  These  include: 

Cost  estimating. 

Performance  measurement. 

Decision  support  systems. 

Rapid  prototyping. 

Recent  INPUT  studies  have  discussed  and  recommended  initiatives  in  these 
areas: 

Performance  Measurement  and  Capacity  Planning,  June  1981. 

Decision  Support  Systems,  September  1981  (Decision  Support  Systems 
and  Prototyping). 

Budgeting  and  Planning  Techniques,  June  1982  (Cost  estimating  and 
prototyping). 
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CATALOG  NO. 


iTTlT.lnl.ql  I  I 


EVALUATING  LEVELS  OF  SERVICE:  EDP  DIRECTOR  QUESTIONNAIRE 

We  are  doing  a  study  on  how  data  processing  service  levels  for  user  departments 

are  established,  measured  and  evaluated,  i.e.,  how  well  EDP  departments  are  meeting 

user  needs  and  what  plans  exist  for  improvement. 

I.        First  of  all,  I  would  like  to  get  an  understanding  of  your  data  processing  operation. 

How  large  is  your  company's  overall  business  and  what  portion  of  it  does  your 
own  EDP  operation  support? 

DOLLAR         NUMBER  OF      NUMBER  OF 
REVENUES        LOCATIONS  EMPLOYEES 

Total  Company  Size       

Portion  Supported       

•  What  is  your  1982  EDP  budget  =  $  million 

•  What  mainframes  do  you  have: 

Manufacturer/Model  Number 


•         What  is  the  approximate  percent  of  cycles  that  are  used  to  support 
on-line  systems?   % 


Used  by  the  EDP  department? 
Used  by  user  departments? 
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About  what  percent  of  your  programmer/analyst  resources  are  devoted  to 
developing  new  systems  (including  major  rewrites  or  enhancements  to  existing 
systems)?  % 

•  What  was  this  percentage  two  years  ago?   % 

•  What  percentage  do  you  expect  two  years  from  now? 

 % 

In  some  of  the  questions  that  I  am  going  to  ask  you,  I'll  be  contrasting  what 
happens  concerning  all  your  systems  generally  with  what  happens  in  regard 
to  your  key  (i.e.,  most  important)  system  or  function. 

•  What  system  or  function  do  you  consider  to  be  the  key  one? 


Why  is  it  considered  to  be  the  most  important? 

How  large  is  it? 

In  terms  of  approximate  percent  of  machine  resources  used? 
 % 

Using  other  measures. 

Which  department(s)  is  the  chief  user  of  this  application? 

When  was  this  system  first  installed? 

About  how  many  major  enhancements  or  extensions  have  taken  place? 
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What  is  the  system  uptime  percent  for  your  computer  system  generally  during 
the  prime  shift?  How  would  you  rate  the  acceptability  of  this  uptime  percent, 
both  from  your  standpoint  and  from  that  of  your  users?  What  do  you  think 
the  situation  was  two  years  ago  and  what  do  you  think  it  will  be  two  years 
from  now? 

Level  Of  Acceptability* 

EDP  '  User 

Percent         Department  Departments 

Now?       

Two  years  ago?       

Two  years  from  now?       

*l  -  Low;  3  =  Medium;  5  =  High 

•         What  kind  of  measurements  do  you  use  to  track  this  data?  Describe. 


What  improvements  do  you  have  planned? 


r 
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5b.      Please  describe  any  differences  for  your  key  systenn? 


5c.      Please  estimate  the  average  uptime  percent  for  an  average  user  terminal 
during  the  hours  needed?  (This  would  include  the  uptime  for  the  computer 
system  in  "a"  above  as  well  as  that  for  communication  controllers,  trunk  and 
secondary  communication  lines,  modems,  individual  terminals,  etc.) 
% 
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7a.      About  how  much  movement  annually,  both  temporary  and  permanent,  is  there 
between  the  EDP  department  and  user  departments,  expressed  as  a  percentage 
of  total  EDP  personnel. 


Annual  Movement,  As  Percent 
 Of  EDP  Staff  

Rotation  Transfer 
(i.e.,  Temporary)  (i.e..  Permanent) 

Into  EDP  Department     


Out  of  EDP  Department 


•        What  is  the  general  level/title  of  such  people? 


7b.      About  how  much  more  (or  less)  of  such  movement  should  there  be? 


Why? 


8.        What  level  of  service  do  you  think  the  EDP  department  is  providing  to  user 
departments  in  general  and  for  the  key  system?  Why?  What  level  of  service 
do  they  think  they  are  receiving,  in  your  opinion?  Why? 
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What  do  you  think  these  ratings  were  two  years  ago? 


Level  Of  Service 


Rating* 


Two 

Now      Years  Ago 


Comment 


EDP  view 


All  user  view 


Key  user  view 


9.        How  would  you  rote  EDP  department  performance  now  and  two  years  ago 
in  the  following  areas,  both  generally  and  for  the  key  system? 


All  Systems* 
Two 

Now      Years  Ago 


Key  System* 
Two 

Now      Years  Ago 


Documentation 
Training 

System  User  Friendliness 

Customer  Service 
(e.g.,  Responding  to 
"hot  line"  calls) 

System  Quality 

Correctness  of  Programs 

Meeting  Schedules 


*I  =  Low;  3  =  Medium;  5  =  High 

10.       I  would  like  to  learn  the  extent  that  user  departments  are  using  data  processing 
services  other  than  that  supplied  by  the  company's  central  EDP  department. 


lOa.     Do  user  departments  have  to  seek  the  approval  of  the  central  EDP  department 
before  using  another  source  of  service? 


(    )  YES        (    )  NO 

-92- 

©1982  by  INPUT.  Reproduction  Prohibited. 


INPI 


CATALOG  NO. 


M  I  I  I  I  I 


If  yes,  go  to  I  Ob;  if  no,  go  to  lOc. 
Why?  

I  Ob.     If  yes,  does  the  process  work?(Describe) 


Is  approval  generally  given? 

How  often  do  user  departments  ignore  this  process? 

lOc.     If  no,  do  you  track  how  much  user  departments  use  another  source  of  service? 
(   )YES        (  )N0 

If  yes,  about  how  accurate  is  the  tracking  process  in  terms  of  tracking  dollars 
+  or  -? 

 % 

If  no,  why  not? 
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lOe.     What  level  of  assistance  do  you  provide  user  departments  in  selecting  or  managing 
the  use  of  such  outside  sources?  (Prompt  from  list) 


1  Alternative 
1        External  Source 

Level  of 
Assistance* 

Comment 

1   Outside  timesharing 
(e.g.,  financial  modeling 
software) 

Turnkey  systems 

 1  II 

Personal  computer 

Minicomputer  systems 

Outside  consultants 

Department  controlled 
and  paid  programmers/ 
analysis 

* 

Department  controlled  and 
budgeted  mainframes 

*!  =  Low;  3  =  Medium;  5  =  High 
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1  U      Please  tell  me  the  two  biggest  problems  you  have  in  dealing  with  user  departments? 
Why  are  these  issues  problems?  What  are  you  doing  to  deal  with  them? 


Problem 

Reasons 

Solution  1 
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12.  '     How  would  you  rate  the  involvement  of  user  departments  generally  in  making 
the  major  decisions  in  what  a  new  system  (or  major  enhancement)  will  contain 
(i.e.,  system  specifications)? 

•  How  satisfied  would  you  say  that  you  are  with  this  level  of  involvement? 

Users  generally? 
Key  users? 

•  How  does  this  compare  with  the  situation  two  years  ago? 

What  do  you  expect  it  to  be  two  years  from  now? 

Satisfaction  With  This  Involvement* 
Rating**     EDP  Dept.       All  Users       Key  Users 

Now         

Two  years  ago         

Two  years  in  future         

*  =  I  =  Low;  3  =  Medium;  5  =  High 

**l  =  All  by  EDP;  2  =  Most  by  EDP;  3  =  Shared  between  EDP  and  users;  4  =  Most 
by  users;  5  =  All  by  users 
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13.      What  is  the  level  of  user  involvement  in  design  reviews  (for  new  development/major 
enhancements)? 

Satisfaction  With  This  Level* 
Rating**    EDP  Dept.       All  Users       Key  Users 


Now 


Two  years  ago 
Two  years  from  now 


*1  =  Low;  3  =  Medium;  5  =  High 

**l  =  All  by  EDP;  2  =  Most  by  EDP;  3  =  Shared  between  EDP  and  users; 
4  =  Most  by  users;  5  =  All  by  users. 

•         Describe  what  changes  are  planned. 
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1 4a.     Suppose  a  user  department  requests  system  development  work  (either  a  new 

system  or  an  enhancement).  About  how  long  would  it  be  before  (a)  a  feasibility 
study  began  and  (b)  the  actual  system  development  work  began?  Assume 
that  there  are  four  levels  of  ultimate  effort  involved: 


•  Several  man-days,  or 

•  Several  man-months,  or 

•  Several  man-years,  or 

•  Many  man-years. 

About  how  long  would  it  be  before  a  feasibility  study  began  (if  any)  and  the 
actual  systems  development  work  began? 


Level  of  Effort 

Months  Before  Work  Begins  On 

Comments 

Feasibility 
Study 

Systems 
Development 

Several  Man-Days 

Several  Man-Months 

Several  Man-Years 

Many  Man-Years 

14b.     How  satisfied  is  the  EDP  department,  user  departments  generally  and  the 
key  user  departments  with  these  time  periods?  Why? 


1     EDP  Department 

Satisfaction* 

Reason 

All  Users 

Key  Users 
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•  What  changes  are  planned? 

1 5a.     Do  you  (or  have  you)  used  a  chargeback  system? 

(   )YES        (  )N0 
15b.     If  yes: 

•  How  long  have  you  used  it? 

•  What  kind  of  units  are  used  to  establish  charges? 

Resource  usage  units  (e.g.,  CPU  cycles,  disk  tracks,  etc.) 

Output  units  (e.g.,  per  report,  per  transaction,  etc.) 
How  are  development  and  maintenance  costs  charged? 


•         Is  the  chargeback  system  primarily  administrative  or  financial  in  nature; 
that  is: 

If  a  department  overruns  does  it  need  administrative  approval 
as  opposed  to  a  change  in  its  budget?  (Describe) 


Can  budgeted  EDP  dollars  be  spent  on  either  internal  or  external 
data  processing  services  as  the  user  department  sees  fit?  (Describe) 
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Can  unspent  chargeback  dollars  be  used  for  non-data  processing 
expenditures  (or  be  switched  around  as  easily  as  any  other  budget 
categories)?  (Describe) 


•         In  general,  how  well  has  your  chargeback  system  worked  and  why? 


Satisfaction* 

Reason 

From  the  standpoint  of: 

The  EDP  department? 

User  departments  in 
general? 

The  department(s) 
concerned  with  the 
key  system? 

*l  =  Low;  3  =  Medium;  5  =  High. 


What  changes  do  you  plan  to  make  to  the  chargeback  system 
and  why? 
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16a.     Do  you  measure  user  satisfaction? 

(   )YES        (  )N0 
16b.     If  yes: 

•  How  long  have  you  done  so? 

•  What  is  the  process? 


•         What  are  sonne  of  the  important  things  you  have  learned? 


•         What  changes  have  you  made  to  your  operation  as  a  result? 


•         What  changes  do  you  expect  to  make  in  your  process?  Why? 


16c.     If  no: 

•        Why  not? 
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Do  you  plan  to?  Why? 


1 7a.    Have  you  used  or  are  considering  using  so-called  "user  contracts"  with  your 
users? 

(   )  YES        (   )  NO  (Go  to  I7d) 
17b.     If  yes: 

•         Who  took  the  initiative  and  why? 


•         Please  describe  what  is/will  be  included? 
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17c.     If  now  used: 

•         How  long  have  they  been  used? 


•         What  is  the  level  of  satisfaction  (and  why)  among  the  parties  involved? 


Party 

Satisfaction* 

Reason 

EDP  Departnnent 

User  Departments  Generally 

Departments  Using  Most 
Important  System 

*l  =  Low;  3  =  Medium;  5  -  High 


•         What  changes  do  you  have  planned  and  why? 
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1 7d.     If  no: 

- .  -    •         Why  haven't  you  considered  using  a  user  contract? 


•         Under  what  conditions  might  you  consider  using  a  user  contract? 


18.      Service  satisfaction  can  be  viewed  as  having  two  components. 

•  Objective,  quantifiable  factors,  such  as  response  time,  system  reliability, 
etc. 

•  Qualitative  factors,  such  as  attitudes,  personalities,  etc. 

In  your  organization  how  important  would  you  say  that  each  of  these  factors 
is  in  determining  satisfaction  with  EDP  levels  of  service?  How  do  you  expert 
the  importance  to  change  in  the  future? 


Importance* 

Change  Expected 

Quantitative 

Qualitative 

*l  =  Low;  3  =  Medium;  5  =  High 
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19.      What  other  plans,  not  touched  on  here,  do  you  have  for  improving  service 
to  user  departments? 


20.      In  some  cases  we  plan  on  interviewing  user  personnel  also.  Can  you  suggest 
one  or  more  users  who  would  be  quite  knowledgeable  about  your  key  system? 

Name:  

Title:  

Department:  

Telephone  (If  available):  

Name:  

Title:  

Department:  

Telephone  (If  available):  
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EVALUATING  LEVELS  OF  SERVICE:  EDP  SERVICE  USER 


We  are  doing  a  study  on  how  data  processing  service  levels  for  user  departments 

are  established,  measured,  and  evaluated,  i.e.,  how  well  EDP  departments  are  meeting 

user  needs  and  what  plans  exist  for  improvement. 

I.        First  of  all,  I  would  like  to  get  an  understanding  of  your  department/division's 
operations. 

•  Number  of  employees:  

•  Number  of  locations: 


•  Department  budget:  $  million 

•  Your  spending  on  EDP  services. 

Chargeback/al location  for  corporate  EDP  services: 
$  million 

Spending  on  outside  EDP  service,  consultants,  contractors,  etc. 
$  million 

Spending  from  your  own  budget  for  EDP  specialists  on  company 
payroll  and  for  your  own  EDP  equipment. 
$  million 

Total  $  million 
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2,        What  data  processing  activity  do  you  consider  to  be  your  key  (i.e.,  most  important) 
system  or  function? 

•         Why  is  it  considered  to  be  the  most  important? 


•  How  large  is  it? 

Using  other  measures. 

•  About  how  many  major  enhancements  or  extensions  have  taken  place? 

•  What  improvements  are  you  expecting  to  make  in  the  system? 


3a.      How  important  to  your  key  EDP  system  is  having  computer  terminals  that 

are  available  at  all  times  during  your  working  hours?  (*l  =  Low;  3  =  Medium; 
5  =  High) 

* 

3b.      About  what  percent  of  the  time  would  you  say  that  the  typical  terminal  on 

your  key  system  is  actually  working  and  available  (i.e.,  is  "up")?   % 

•  Is  this  satisfactory: 

(   )YES        (  )N0 

Why?  

•  What  actions  are  you  and/or  the  EDP  department  taking  to  improve 
it? 
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About  how  much  movement  annually,  both  temporary  and  permanent,  is  there 
between  the  EDP  department  and  your  department? 

Annual  Movement 

Rotation  Transfer 
(i.e.,  Temporary)  (i.e.,  Permanent) 


Into  EDP  Department 
Out  of  EDP  Department 


What  is  the  general  level/title  of  such  people? 


About  how  much  more  (or  less)  of  such  movement  should  there  be? 


Why' 


What  level  of  service  do  you  think  the  EDP  department  is  providing  to  user 
departments  in  general  and  for  your  key  system?  Why? 
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What  do  you  think  these  ratings  were  two  years  ago? 


Rating* 


Level  Of  Service 


Two 

Now     Years  Ago 


Conament 


All  user  view 


Key  user  view 


7. 


How  would  you  rate  EDP  department  perfornnance  now  and  two  years  ago 
in  the  following  areas,  both  generally  and  for  your  key  system? 


Ail  Systems* 
Two 

Now      Years  Ago 


Key  System* 
Two 

Now     Years  Ago 


Documentation         

Training         

System  User  Friendliness      ____   

Customer  Service 
(e.g.,  Responding  to 

"hot  line"  calls)         

System  Quality         

Correctness  of  Program         

Meeting  Schedules         

*l  =  Low;  3  =  Medium;  5  =  High 

8.        I  would  like  to  learn  the  extent  that  user  departments  are  using  data  processing 
services  other  than  that  supplied  by  the  company's  central  EDP  department. 

8a.      Do  user  departments  have  to  seek  the  approval  of  the  central  EDP  department 
before  using  another  source  of  service? 


(  )YES 


(  )N0 
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If  yes,  go  to  8b. 


Why? 


8b.      If  yes,  does  the  process  work?(Describe) 


Is  approval  generally  given? 


How  often  do  user  departments  ignore  this  process? 
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lOa.     What  level  of  assistance  does  the  EDP  department  provide  your  department 
in  selecting  or  managing  the  use  of  such  outside  sources?  (Prompt) 


Alternative 
External  Source 

Level  of 
Assistance* 

Comment 

Outside  transaction 
processing  (e.g.,  payroll; 

Outside  timesharing 
(e.g.,  financial  modeling 
software) 

Turnkey  systems 

Personal  computer 

systems/minicomputer 

systems 

Outside  consultants 

Department  controlled 
and  paid  programmers/ 
analysis 

Department  controlled  and 
budgeted  mainframes 

*l  =  Low;  3  =  Medium;  5  =  High 


I  Ob.     If  the  EDP  department  offered  in-house  alternatives  to  your  current  outside 

sources  of  data  processing  services,  would  you  see  it  as  an  advantage  or  disadvantages? 
^.  Why? 
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I  i.      Please  tell  me  the  two  biggest  problems  you  have  in  dealing  with  the  EDP 
department? 

Why  are  these  issues  problems?  What  is  being  done  to  deal  with  them? 


Problem 

Reasons 

Solution 
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12.      How  would  you  rate  the  involvement  of  your  department  in  making  the  major 
decisions  in  what  a  new  system  (or  major  enhancement)  will  contain  (i.e., 
system  specifications)? 

•  How  satisfied  would  you  say  that  you  are  with  this  level  of  involvement? 

•  How  does  this  compare  with  the  situation  two  years  ago? 

What  do  you  expect  it  to  be  two  years  from  now? 

Satisfaction  With  This  Involvement* 
Rating**  Key  Users 

Now     

Two  years  ago     

Two  years  in  future     


*l  =  Low;  3  =  Medium;  5  =  High 

**l  =  All  by  EDP;  2  =  Most  by  EDP;  3  =  Shared  between  EDP  and  users;  4  =  Most 
by  users;  5  =  All  by  users 
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13.      What  Is  the  level  of  your  department's  involvement  in  design  reviews  (for 
new  development/major  enhancements)? 

Satisfaction  With  This  Level* 
Rating**  Key  Users 

Now  '   

Two  years  ago     

Two  years  from  now     


*l  =  Low;  3  =  Medium;  5  =  High 

**l  =  All  by  EDP;  2  =  Most  by  EDP;  3  =  Shared  between  EDP  and  users; 
4  =  Most  by  users;  5  =  All  by  users. 

•         Describe  what  changes  are  planned. 
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14a.     Suppose  a  user  department  requests  system  development  work  (either  a  new 

system  or  an  enhancement).  About  how  long  would  it  be  before  (a)  a  feasibility 
study  began  and  (b)  the  actual  system  development  work  began?  Assume 
that  there  are  four  levels  of  ultimate  effort  involved: 


•  Several  man-days,  or 

•  Several  man-months,  or 

•  Several  man-years,  or 

•  Many  man-years. 

About  how  long  would  it  be  before  a  feasibility  study  began  (if  any)  and  the 
actual  systems  development  work  began? 


Level  of  Effort 

Months  Before  Work  Begins  On 

Comments 

Feasibi  lity 
Study 

Systems 
Development 

Several  Man-Days 

Several  Man-Months 

Several  Man-Years 

Many  Man-Years 

/ 


14b.     How  satisfied  are  you  with  these  time  periods?  Why? 


Satisfaction* 

Reason 

Key  Users 

I 
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•  What  changes  are  planned? 

1 5a.    Does  your  department  use  (or  have  you)  used  a  chargeback  system  where  your 
department  Is  charged  by  corporate  EDP  for  data  processing  services? 

(   )YES        (  )N0 

15b.     if  yes: 

•  How  long  have  you  used  it? 


What  kind  of  units  are  used  to  establish  charges? 

Resource  usage  units  (e.g.,  CPU  cycles,  disk  tracks,  etc.) 

Output  units  (e.g.,  per  report,  per  transaction,  etc.) 

How  are  development  and  maintenance  costs  charged? 


•         Is  the  chargeback  system  primarily  administrative  or  financial  In  nature; 
that  is: 

If  a  department  overruns  does  it  need  administrative  approval 
as  opposed  to  a  change  in  its  budget?  (Describe) 
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Can  budgeted  EDP  dollars  be  spent  on  either  internal  or  external 
data  processing  services  as  the  user  department  sees  fit?  (Describe) 


Can  unspent  chargeback  dollars  be  used  for  non-data  processing 
expenditures  (or  be  switched  around  as  easily  as  any  other  budget 
categories)?  (Describe) 


•         In  general,  how  well  has  your  chargeback  system  worked  and  why? 


Satisfaction* 

Reason 

From  the  standpoint  of: 

The  department(s) 
concerned  with  the 
key  system? 

*l  =  Low;  3  =  Medium;  5  =  High.- 


What  changes  do  you  plan  to  make  to  the  chargeback  system 
and  why? 
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16a.     Is  user  department  satisfaction  with  corporate  data  processing  service  measured 


in  any  way  r 


(   )YES        (  )N0 


16b.     if  yes: 


How  long  has  it  been  going  on? 


•         What  is  the  process? 


•         What  changes  have  been  made  as  a  result? 


•         What  changes  do  you  think  should  be  made  to  the  process?  Why? 


1 6c.     If  no: 


•         Do  you  think  a  regular  process  of  measuring  user  satisfaction  would 
be  useful?  Why? 
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17a.     Does  your  company  use  or  is  it  considering  using  so-called  "user  contracts" 
between  the  EDP  and  users? 

(   )  YES        (   )  NO  (Go  to  i7d) 

17b.     If  yes: 

•         Who  took  the  initiative  and  why? 


Please  describe  what  is/will  be  included? 


i 


i 
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1 7c.     If  now  used: 


How  long  have  they  been  used? 


What  is  the  level  of  satisfaction  (and  why)  among  thq  parties  involved? 


Party 

Satisfaction* 

• 

Reason 

EDP  Department 

User  Departments  Generally 

Departments  Using  Most 
important  System 

*l  =  Low;  3  =  Medium;  5  =  High 

•         What  changes  do  you  have  planned  and  why? 
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I7d.     If  no: 


Do  you  think  it  would  be  useful?  Why? 


...  l-.^<;<„   

18.      Service  satisfaction  can  be  viewed  as  having  two  components. 

•  Objective,  quantifiable  factors,  such  as  response  time,  system  reliability, 
etc. 

•  Qualitative  factors,  such  as  attitudes,  personalities,  etc. 

In  your  organization  how  important  would  you  say  that  each  of  these  factors 
is  in  determining  satisfaction  with  EDP  levels  of  service?  How  do  you  expert 
the  importance  to  change  in  the  future? 


Importance* 

Change  Expected 

Quantitative 

Qualitative 

*l  =  Low;  3  =  Medium;  5  =  High 

I?.       In  what  other  ways,  not  touched  on  here,  do  yon  think  that  the  kind  of  data 
processing  service  you  receive  from  corporate  EDP  could  be  improved? 
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APPENDIX  Ci  DETAILED  RESPONDENT  PROFILES 


EXHIBIT  C-1 


REVENUE  SIZE  OF  COMPANIES  INTERVIEWED 


SIZE  (REVENUES) 

NUMBER  OF 
COMPANIES 
INTERVIEWED 

Diiiion  &  over 

li 

H 

$2.1  -  2.5  billion 

2 

$1.1  -  2.0  billion 

8 

$501  million  -  1  billion 

3 

$201  -  500  million 

3 

$101  -  200  million 

3 

Under  $100  million 

5 

TOTAL 

28 
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EMPLOYEE  SIZE  OF  COMPANIES  INTERVIEWED 


SIZE 

(NUMBER  OF  EMPLOYEES) 

Nl  IMRFR  OF 

COMPANIES 
INTERVIEWED 

Over  30,  000 

3 

20, 000  -  30,000 

4 

10,  000  -  1  9,999 

4 

6,000  -  9,999 

5 

1,000  -  5,999 

9 

Under  1,000 

3 

TOTAL 

28 
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NUMBER  AND  TYPES  OF  COMPANIES  INTERVIEWED 


TYPE  OF  COMPANY 

NUMBER  OF 
COMPANIES 
INTERVIEWED 

Discrete  Manufacturing 

7 

Process  Manufacturing 

8 

Insurance 

5 

Banking  and  Finance 

2 

Transportation 

2 

Utilities 

2 

Services 

2 

TOTAL 

28 

-  129  - 

©1982  by  INPUT.  Reproduction  Prohibited. 


-130- 


APPENDIX  D:  SAMPLE  USER  SATISFACTION  SURVEY  FORM 


APPENDIX  D 


USER  SATISFACTION  SURVEY 


The  Information  Systems  department  is  seeking  your  help  in  evaluating  the  quality 
of  service  and  support  it  is  providing.  Your  answers  to  this  questionnaire  will  assist 
in  determining  opportunities  for  improving  our  value  to  you.  Our  objective  is  to  pro- 
vide the  information  you  need  when  you  need  it,  and  at  the  least  possible  cost  to  you. 


What  is  your  opinion  of  the  service  you  received  in  each  of  the  following  areas  during 
the  past  six  months?   Please  indicate  your  answers  to  each  question  by  circling  the 
appropriate  number.    "5"  indicates  the  highest  level  of  satisfaction,    "3"  is  average, 
and  "1"  is  dissatisfaction.    If  not  applicable,  please  circle  "0". 

If  you  have  any  questions  about  the  survey  please  call   

at 


I.     DEVELOPMENT,  ENHANCEMENT,  MAINTENANCE 


la.  Did  the  systems  team  begin  work  on  your  applica- 
tion in  a  timely  manner? 

lb.  Which  application(s)  


*  New  Development    0    12    3    4  5 
Enhancement  0    1    2    3    U  5 

Maintenance  0    1    2    3    4  5 


2.  Were  your  applications  delivered  on  schedule? 


New  Development  0  1  2  3  4  5 
Enhancement  0    1    2    3    4  5 

Maintenance  0    1    2    3    4  5 


3.  Were  your  applications  delivered  on  or  under 
budget,  or  were  you  prepared  well  in  advance  for 
any  overruns? 

4.  Was  the  systems  team  willing  to  accept  your  influ- 
ence in  the  design  of  your  application. 

5.  Did  the  systems  team  understand  and  incorporate 
your  business  needs  into  the  application  design? 

6.  Could  the  systems  team  explain  technical  matters  in 
terms  you  can  understand? 

7.  How  well  did  the  systems  team  support  and  balance 
your  requirements  against  technical  limitations? 

*  Definitions:  New  Development  =  brand  new  or  replacennent  system. 

Enhancement  =  voluntary  improvement  providing  additional  capabilities. 
Maintenance  =  mandatory  change,  or  repair  of  errors. 


0    1    2    3    4  5 


0    1    2    3    4  5 


0    1    2    3    4  5 


0    1    2    3    4  5 


0    1    2    3    4  5 
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8.  Did  the  systems  team  seek  to  design  and  build 
flexible  applications  in  anticipation  of  future  needs, 

e.g.,  volume  increases?  0    12    3   4  5 

9.  How  well  did  the  systems  team  integrate  the  appli- 
cation within  the  total  system?  0    1    2    3   U  5 

10.  Did  the  systems  team  deliver  agreed-upon  system 

capability  (output,  controls,  performance)?  0    12    3   4  5 


11.  What  are  the  shortfalls? 


0    12    3    4  5 


12.  Documentation  (availability,  currentness,  ease  of      New  Development  0    1    2    3    4  5 
use) .  After  changes        0    1    2    3    4  5 

13.  Ability  to  operate  without  further  dependence  on 

the  systems  team.  0    1    2    3   4  5 

14.  What  is  your  level  of  satisfaction  with  hand-over  and 
initial  implementation  period  of  a  new  or  changed 

application?  0    1    2    3   4  5 

15.  How  well-tested  was  the  system?  0    1    2    3   4  5 

16.  Were  you  adequately  assisted  in  developing  the  cost 

benefit  calculation?  0    1    2    3   4  5 

17.  Are  you  satisfied  with  the  priority  given  to  new 

applications?  0    1    2    3   4  5 

18.  How  do  you  rate  the  return-on-investment  for  the     New  Development  0    1    2    3    4  5 
cost  of:  Enhancement        0    1    2    3    4  5 

Maintenance  0    1    2    3    4  5 

19.  Please  rank  in  order  of  importance  to  you  the  services 
listed  in  part  I . 

Most  important  -  question  number   . 

Second  -  question  number   . 

Third  -  question  number   . 

Fourth  -  question  number   . 

Fifth  -  question  number   . 

20.  Any  other  comments  on  the  applications  software  aspects?   
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II.    DATA  CENTER  ACTIVITY 

A.       USE  OF  TERMINAL 

21a.    Access  to  the  system  when  you  want  to  use  it.  0    12    3   4  5 

21b.    What  is  your      measure  EUestimate  of  availability?  % 

22a.    Terminal  response  time.  0    12    3   4  5 

22b.    What  is  your  Qmeasure  [ZJestimate  of  response  time? 

Range:   to   seconds.  Average:   seconds. 

23.  Down-time  warning  procedure.  0    1    2    3   4  5 

24.  Screen-formating  procedure.  0    1    2    3   4  5 

25.  Ease  of  use  of  input/output  devices.  0    1    2    3   4  5 

26.  Where  is  your  terminal  located?   

 (location/building) 


27a.     How  many  hours /week  do  you  use  your  terminal?   hours/ 

week. 

27b.    During  what  time  is  your  main  use?       :       am.  /pm. 
B.       BATCH  PROCESSING 

28.  Ability  of  Data  Center  to  meet  delivery  schedule  for  output.     0    1    2   3   4  5 

29.  Correctness  of  data  (accuracy  in  processing,  adequacy  of 

batch  controls,  other  data  center  control  procedures).  0    1    2    3   4  5 

30.  Level  of  detail,  summarization  and  format  of  data.  0    1    2    3   4  5 

31.  Privacy  controls  (individual  or  Corporate).  0    1    2    3   4  5 

32.  Documentation  (availability,  currentness,  ease  of  use) .  0    1    2    3   4  5 

33.  Resolution  of  problems  (initial  response,  on-going  communi- 
cation, long-standing  unresoved  problems).  0    1    2    3   4  5 

34.  Contingency  planning  (back-up  files,  alternate  access  during 

or  after  down-time) .  0    1    2    3   4  5 
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35.  User  training  (equipment.  Data  Center  procedures)  0  1    2  3  4  5 

36.  Value  received  for  the  cost  of  the  operation  service.  0  1    2  3  U  5 

37.  Current  charge-back  "formula"  0  12  3  4  5 

38a.  Overall  quality  of  service  from  your  Data  Center.  0  1    2  3  4  5 

38b.  Which  Data  Center  do  you  use?   0  1    2  3  4  5 

39.     Please  rank  in  order  of  importance  to  you  the  services  listed 
in  part  11. 

Most  important  -  question  number   . 

Second  -  question  number   . 

Third  -  question  number   . 

Fourth  -  question  number   . 

Fifth  -  question  number   . 


III.  OPTIONAL  QUESTIONS  ABOUT  YOURSELF 
40.    Your  Name:  Title 


Department:   Phone:  (  ) 


41.  How  many  years  MIS  and /or  user  experience  overall  do  you  have? 
Less  than  1  yearj    |    1  year  to  3  years  \    \  More  than  3  years  I  I 

42.  Please  check  this  box  if  you  are  a  Manager  mainly  involved  in  planning!  | 

43.  Please  check  this  box  if  you  are  a  Manager  or  Supervisor  mainly  involved  in 
day-to-day  control]  | 

Comments : 


Please  return  before       /      /  to 


If  you  have  any  questions  about  the  survey  please  call  J  )_ 
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